Cloud Gaming Device Handover

ABSTRACT

Methods and systems to identify options of secondary client devices for a device handover of game play include establishing a session of game play of a game for a primary client device by executing the game on a server and streaming video frames to the primary client device. A profile is generated for each secondary client device that is identified to be local to the primary client device. A handover option is provided to the primary client device during game play, identifying one or more of the secondary client devices, based on the profile. A selection of a secondary client device from the handover option causes pausing of the streaming of video frames to the primary client device, accessing the game state for the game, and resuming game play using the secondary client device.

CLAIM OF PRIORITY

The present application is a Continuation of U.S. patent applicationSer. No. 17/688,723, filed on Mar. 7, 2022, and entitled, “Cloud GamingDevice Handover,” which is a continuation of U.S. patent applicationSer. No. 16/812,195, filed on Mar. 6, 2020, (since issued as U.S. Pat.No. 11,285,386 on Mar. 29, 2022) and entitled, “Cloud Gaming DeviceHandover,” which is a Continuation of U.S. patent application Ser. No.16/365,560, filed on Mar. 26, 2019, (since issued as U.S. Pat. No.10,898,800 on Jan. 26, 2021) and entitled, “Cloud Gaming DeviceHandover,” which is a continuation of U.S. patent application Ser. No.15/141,799, filed on Apr. 28, 2016, (since issued as U.S. Pat. No.10,238,965 on Mar. 26, 2019) entitled “Cloud Gaming Device Handover”,which are incorporated herein by reference in their entirety.

FIELD

The present disclosure relates to systems and methods for providinggames for user interaction, and more specifically to providing optionsfor selecting alternate client devices for a device handover duringcloud gaming.

BACKGROUND Description of the Related Art

One of the rapidly growing technologies is in the field of cloud gamingwhere users are able to access a number of games available on a cloudgaming site over a network, such as the Internet, and begin playing thegame. A user accesses his/her account on the cloud gaming site andselects a game from a list of games that are available for the useraccount, for game play. When the user selects a game for game play fromthe cloud gaming site, using a client device, a server in the cloudgaming site starts a session of game play and begins streaming the videoframes of the game to the client device from which the user accessed thecloud gaming site. During the session, if the user wishes to switch to asecond client device for playing the game, the server on the cloudgaming site has to restart the game from the beginning so that it canstart streaming the video frames to the second client device. The serverhas to conduct bandwidth testing and other quality of service testingbefore it can begin streaming the video frames of the game to the secondclient device. The restarting of the game and pre-testing of the secondclient device leads to unnecessary wait for the user, causing userfrustration.

It is within this context that embodiments of the invention arise.

SUMMARY OF THE INVENTION

Embodiments of the present invention disclose methods, systems andcomputer readable media that are used for identifying options ofalternate client devices for handing over game play of a game hosted bya cloud gaming site. The various embodiments enable a primary clientdevice that is currently used for playing a game hosted by a cloudgaming system on a cloud gaming site, to engage in background profilingof secondary client devices that are identified to be local to theprimary client device. The profiling of the secondary client devices isdone to determine which ones of the secondary client devices arequalified to continue a game play session of the game, and is performedbefore a device handover request is received from the primary clientdevice. The profile identifies the device attributes of each of thesecondary client devices. In some embodiments, the cloud gaming systemmay use the profiles for provisioning services to different secondaryclient devices to get these secondary client devices ready for playingthe game. Each game hosted by the cloud gaming system, may have its ownservice and attribute requirements.

Profiling allows the cloud gaming system to identify the services,device attributes available in each of the secondary client devices, anddetermine the services, device attributes that are not available or arenot up to a level required in order for the respective secondary clientdevices to be used for interacting with the game executing on the cloudgaming site. The cloud gaming system, in some embodiments, may use theprofile of each secondary client device to provision appropriateresources so as to boost the services or device attributes of specificones of the secondary client devices. Such provisioning may be performedby the cloud gaming system ahead of time or in the background during agame play session initiated by a primary client device, in anticipationthat a specific one of the secondary client devices may be selected fora device handover by the primary client device for game play of the gameduring the game play session.

When a request for a device handover is received at the cloud gamingsystem, from the primary client device, the profiling allows the cloudgaming system to perform a faster, frictionless device handover from theprimary client device to a specific secondary client device and thesecondary client device will have the necessary resources (services,device attributes) to continue the game play session. In other words,profiling allows the cloud gaming system to qualify the secondary clientdevices, so that when a handover request is received, the system is ableto switch the client device with which the user interacts with the cloudgaming site for playing the game.

A successful device change would include a server of the cloud gamingsystem that is executing the game, to pause game play and stop streamingof the video frames for the game to the primary client device, save agame state of the game, and provide an option for resuming the game playof the game to a secondary client device identified by the handoveroption. When a resume request is received from the secondary clientdevice, the server on the cloud gaming system retrieves the game state,resumes the game play from where the game was paused, and continuesstreaming the video frames for the game to the secondary client device.

Discovering and profiling the secondary client devices ahead of timesaves considerable wait time during device handover. The secondaryclient devices that are identified in the handover option are qualifiedto provide a comparable game play experience as that experienced whenusing the primary client device. Further, the profiling and provisioningensures that the video frames are provided to the user at the secondaryclient device with minimal latency. Pausing the game and/or saving thegame state during device handover allows for faster resumption of gameplay of the game.

In one embodiment, a method is provided. The method is executed by aserver of a cloud gaming system and is used to identify options ofsecondary client devices that can be used for device handover of gameplay to one of the secondary client devices. The method includesestablishing a session of game play of a game for a primary clientdevice, by the server, in response to a request received from theprimary client device to play the game. The server of the cloud gamingsystem is configured to execute the game and stream video frames to theprimary client device. A request is received by the server to generate aprofile of one or more secondary client devices that are identified tobe local to the primary client device. The profile is configured toidentify device attributes for the secondary client devices. During gameplay, a handover option is provided by the server to the primary clientdevice. The handover option is configured to identify one or more of thesecondary client devices for a device handover, based on their profile.A selection of a particular secondary client device identified by thehandover option is received from the primary client device. Theselection is configured to pause the streaming of video frames to theprimary client device and save a game state for the game. The pause isconfigured to provide an option to resume play of the game on theparticular secondary client device. A resume request is received fromthe particular secondary client device to continue the play of the gameon the secondary client device. The resume request is configured tocause the server to access the game state for the game and to continuestreaming video frames to the particular secondary client device.

In some embodiments, the request to generate a profile is receivedduring the session.

In some embodiments, the particular secondary client device isdesignated as the primary client device, for the reminder of thesession.

In some embodiments, the request to generate a profile is triggered inresponse to occurrence of a trigger event.

In some embodiments, the request to profile is received from each of thesecondary client devices that are identified to be local to the primaryclient device.

In some embodiments, the device attributes used for qualifying thesecondary client devices are ranked in accordance to requirements of thegame. The secondary client devices provided in the handover option areorganized in accordance to the ranking of the device attributes.

In some embodiments, the secondary client devices are identified usingone or more service discovery protocols implemented within the primaryclient device and the secondary client devices.

In some embodiments, the request to generate the profile is receivedfrom the primary client device. The request includes device identifiersof the secondary client devices for which the profile is to begenerated.

In another embodiment, a method is executed by a server of a cloudgaming system. The method is executed to identify options of secondaryclient devices that can be used for a device handover of game play toone of the secondary client devices. The method includes establishing asession of game play of a game for a primary client device, in responseto a request from the primary client device to play the game. The serverof the cloud gaming system is configured to execute the game and streamvideo frames to the primary client device. One or more secondary clientdevices are detected that are identified to be local to the primaryclient device. Profile is generated for the primary client device andthe one or more of the secondary client devices that are identified tobe local to the primary client device. The profile is configured toidentify device attributes of each of the primary and the secondaryclient devices. A handover option is provided by the server to theprimary client device during the game play. The handover option isconfigured to identify one or more secondary client devices that arelocal to the primary client device and having device attributes that arecapable of enhancing game play experience for a user, based on theprofiles generated for the primary and secondary client devices. Aselection of a particular secondary client device identified by thehandover option is received by the server from the primary clientdevice. The selection is configured to pause the streaming video framesto the primary client device and save a game state for the game. Thepause is configured to provide a resume option for resuming play of thegame on the particular secondary client device identified by thehandover option. A resume request is received by the server from theparticular secondary client device to continue the session of game playon the secondary client device. The resume request is configured tocause the server to retrieve the game state for the game and continuestreaming video frames to the particular secondary client device.

In some embodiments, the profile of the primary and the one or moresecondary client devices are generated in response to a trigger event.

In some embodiments, the profile of the primary and the one or moresecondary client devices are generated in response to a request from theprimary client device.

In yet another embodiment, a cloud gaming system is disclosed. The cloudgaming system includes an application server that is configured toexecute a plurality of games. A game execution engine in the applicationserver receives a request to play a game from a primary client device.In response, the game execution engine executes an instance of the gameand generates video frames for the game. The application server isconfigured to process the generated video frames, encode the videoframes and stream the video frames to the primary client device. Aclient device profiling module within the application server isconfigured to receive request to generate a profile of one or moresecondary client devices that are identified to be local to the primaryclient device, wherein the one or more secondary client devices areidentified based on the profile of the primary client device. The clientdevice profiling module is further configured to generate a list of thesecondary client devices. A handover manager within the applicationserver is configured to obtain the list of secondary client devices,qualify the secondary client devices for playing the game, and forward arefined list of qualified secondary client devices with a handoveroption to the primary client device. The handover manager is furtherconfigured to receive a selection of a particular secondary clientdevice identified by the handover option and, in response, forward asignal to the game execution engine to pause the streaming of the videoframes to the primary client device. In some embodiments, in response tothe pause signal, a game state of the game is also saved. When thehandover manager receives a resume request from the particular secondaryclient device, the handover manager, in response, signals the gameexecution engine to resume game play and continue streaming the videoframes to the particular secondary client device. In the embodimentswherein the game state of the game is saved, the game execution engine,in response to the resume request, may retrieve the game state of thegame and resume game play of the game from a point where it was paused,as indicated by the game state of the game.

In some embodiments, the handover manager is configured to signal apause manager to pause the game play of the game, in response toreceiving a selection of the particular secondary client deviceidentified by a handover option. The pause manager is configured toforward a signal to the game execution engine to pause the game play ofthe game.

In some embodiments, the handover manager is configured to signal anoperating system of the application server to pause transmitting videoframes of the game to the particular primary client device, in responseto selection of the particular secondary client device identified by ahandover option.

In some embodiments, the handover manager is configured to signal aresume manager to resume the game play of the game, in response toreceiving a resume request from the particular secondary client device.The resume manager is configured to forward a signal to the gameexecution engine requesting the game execution engine to retrieve gamestate of the game and to resume streaming video frames of the game tothe particular secondary client device.

In some embodiments, the handover manager is configured to signal aresume manager to resume game play. The resume manager receives thesignal and, in turn, signals the operating system of the applicationserver to resume transmitting video frames of the game to the particularsecondary client device, in response to receiving a resume request fromthe particular secondary client device.

In some embodiments, the handover manager is further configured todesignate the particular secondary client device as the primary clientdevice.

Other aspects and advantages of the invention will become apparent fromthe following detailed description, taken in conjunction with theaccompanying drawings, illustrating by way of example the principles ofthe invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention, together with further advantages thereof, may best beunderstood by reference to the following description taken inconjunction with the accompanying drawings.

FIG. 1 illustrates a simplified block diagram of an example system thatis used to identify options of secondary client device for devicehandover of game play of a game, in accordance with an embodiment of thepresent invention.

FIG. 2 illustrates an example application server with various modulesthat are used for generating a handover option identifying one or moresecondary client devices for device handover, in accordance to anembodiment of the present invention.

FIG. 3 illustrates a primary client device with a switching module thatis used for initiating a device handover, in accordance with oneembodiment of the present invention.

FIG. 4 illustrates an example discovery agent of the switching module inthe primary client device for discovering secondary client devices thatare local to the primary client device, in accordance to an embodimentof the invention.

FIG. 5 illustrates an example client device profiler of a server, suchas an application server, in a cloud gaming system that is used forprofiling the secondary client devices, in accordance with an embodimentof the invention.

FIG. 6A illustrates an example profile generated for the secondaryclient devices that are local to a primary client device, in accordancewith an alternate embodiment of the invention.

FIGS. 6B, 6C-1 and 6C-2 illustrate examples of handover logic at theprimary client device that provides input to select a secondary clientdevice for device handover, in accordance with an embodiment of theinvention.

FIG. 7 illustrates an example flow of operations of a method used foridentifying options of secondary client devices for device handover ofgame play, in accordance with an embodiment of the invention.

FIG. 8 illustrates an example Information Service Provider architecturefor delivering informational content and services to users who aregeographically dispersed and connected via network, in accordance withone embodiment of the present invention.

FIG. 9 illustrates a simplified block diagram of an example Game System,in accordance with various embodiments of the invention.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth inorder to provide a thorough understanding of the present invention. Itwill be apparent, however, to one skilled in the art that the presentinvention may be practiced without some or all of these specificdetails. In other instances, well known process steps have not beendescribed in detail in order not to obscure the present invention.

According to various embodiments of the present invention, a user mayaccess a cloud gaming system through a user account, using a primaryclient device, for selecting a game for game play. The cloud gamingsystem is configured to host a plurality of online games for game playusing resources available to the cloud gaming system. In response to theuser accessing the cloud gaming system, the cloud gaming systemauthenticates the user account using user account information stored ina user accounts database and provides a list of games that are availablefor the user account for game play. The user selection of a game forgame play triggers the cloud gaming system to identify a data centerthat is in the vicinity of a geo location of the primary client deviceand send a request to a server in the data center to execute an instanceof the game and establish a game play session. The server provides thenecessary resources (i.e., processing and communication resources) forexecuting the game, encode video frames of the game and begin streamingthe encoded video frames of the game to the primary client device forrendering and user interaction. User interactions provided at theprimary client device are used to influence game outcome of the gameexecuting on the cloud gaming system. In some embodiments, soon afterinitiating the game play session with the primary client device, aserver of the cloud gaming system may generate a profile of the primaryclient device. The profiling of the primary client device may beinitiated as part of quality of service (QOS) verification to ensurethat bandwidth, latency, and other resource variables of the primaryclient device are sufficient, as part of data center selection andqualification, so that the primary client device can receive and processthe streaming video frames of the game streamed from the server of thedata center. It should be noted that the various embodiments discussedherein may also be extended to generation of audio frames, encoding ofaudio frames, streaming of encoded audio frames for the game to theprimary client device for rendering.

During game play of the game, the primary client device may activelyseek and discover secondary client devices that are local to the primaryclient device and send a signal to each of the secondary client devicesor to select ones of the secondary client devices with a request to getprofiled by a server of the cloud gaming system. The profile may be usedto qualify the secondary client devices for playing the game. Therequest generated by the primary client device identifies the serverthat the secondary client devices have to use to get profiled. Theserver selected for profiling may be the server in the data center thatis executing the game or may be any other server within the same datacenter or in any other data center within the cloud gaming system thatis configured for performing such profiling. The server of the cloudgaming system that is used for profiling includes profile generationlogic to generate a profile of each client device, when requested. Insome embodiments, as and when the primary client device discovers asecondary client device, the discovered secondary client device isprofiled. A handover option is provided at the primary client device,wherein the handover option identifies one or more of the secondaryclient devices that are eligible for a device handover, based on thegenerated profiles of the secondary client devices. In some embodiments,the handover option may be provided for handing over the streaming ofonly the video frames, only the audio frames or both the audio and videoframes of the game. When a user selects a particular secondary clientdevice, the user selection is forwarded to the server, which generates apause signal. The server in the data center, in response to receivingthe pause signal, pauses the game play of the game, and provides aresume option to the particular secondary client device to resume thegame play.

As part of processing the pause signal, the server executing the game atthe data center, in some embodiments, may initiate an event trigger thatcauses the server to save system state at the time when the eventtrigger was initiated. In such embodiments, the system preserves allapplication (including game application) and/or system state withoutdirectly manipulating any game attributes or game variables. The systemstate could be used to re-create the game state of the game. In someembodiments, the system state may be saved using an applicationprogramming interface (API). In alternate embodiments, the server, inresponse to the pause signal, may write out save data of the game at thetime the pause signal was received. The save data records progress madeby the user during game play, which could then be used to re-create acurrent game state of the game, when the game is resumed. For moreinformation on how progression of a user in the game is used tore-create the game state of the game, in one embodiment, reference canbe made to U.S. application Ser. No. 12/917,388, filed on Nov. 1, 2010,and titled, “USER INTERFACE, SYSTEM AND METHOD FOR CONTROLLING A VIDEOSTREAM,” which is incorporated herein by reference in its entirety.

In some embodiments, the user selection of a particular secondary clientdevice indicates that the user is no longer providing game playinteractions for the game from the primary client device. In someembodiments, the user selection of a particular secondary client devicemay indicate that the user no longer wishes to receive at least aportion of streaming game content (i.e., either audio or video frames ofgame content) at the primary client device. For example, the user maynot wish to receive audio portion of the game content at the primaryclient device while still interested in receiving video portion of thegame content at the primary client device. This may be the case if theaudio reception at the primary client device is not at an acceptablelevel, for example. In such embodiments, the audio portion of the gamecontent may be extracted and forwarded to the particular secondaryclient device while the video portion of the game content (with audioportion muted, for example) may continue to be streamed to the primaryclient device. The handover option, in such embodiments, provided at theprimary client device may include options to select appropriate portionof the streaming game content (e.g., video portion, audio portion, etc.)and the particular secondary client device for streaming the selectedportion of the game content as part of the device handover, so that theselected portion of the streaming game content (either the audio or thevideo frames) is redirected to the particular secondary client device.

In some embodiments, in response to receiving the pause signal, theserver in the data center may pause the transmission of the streamingvideo frames and/or streaming audio frames for the game to the primaryclient device. As a result, the game appears to be in a “suspended”state. In this embodiment, the game may continue to be active but maynot be receiving any user interactions till device handover hascompleted.

Selection of resume option at the particular secondary client device isforwarded to the server, leading to the server accessing the game stateof the game (if any) and/or resuming game play of the game from a pointwhere it was paused. The resumed game play causes the video framesand/or audio frames of the game to be streamed to the particularsecondary client device. Alternately, where the transmission of thevideo frames and/or audio frames was paused while the game was keptactive, the resume request would cause the server to resume transmissionof the streaming video frames and/or streaming audio frames of the gameto the particular secondary client device. In some embodiments, thevideo frames and/or audio frames are generated and streamed based onuser interactions received from the particular secondary client device.In embodiments where a portion of the game content is to be streamed tothe primary client device and the remaining portion is to be streamed tothe secondary client device, the video and/or audio frames are generatedand streamed to appropriate client devices (i.e., primary and secondaryclient devices) based on user interactions received from the particularsecondary client device, the primary client device or from both theprimary and secondary client devices.

The secondary client devices identified in the list have the deviceattributes that are required for game play of the game and, in someembodiments, are available for game play during a time when a handoveroption is received. The various embodiments discussed herein provide auser with options of secondary client devices to switch to for playingthe game, during a game play session, and such switch is done in aseamless and frictionless manner. The secondary client devices that areidentified in the options may be discovered using any one of a pluralityof device discovery protocols that are implemented in the clientdevices, based on signals generated by the secondary client devices,based on other devices receiving signals from the secondary clientdevices, etc. The discovered client devices are profiled in thebackground and are qualified for game play of a game so that a devicehandover can be successfully effectuated.

The information provide in the profiles of the secondary client devicesmay be used to not only determine device attributes that are requiredfor game play of a game but also device attributes that are either notavailable at the one or more secondary client devices or are not at alevel that is required for game play of the game. Based on this profileinformation, the cloud gaming system, in some embodiments, may provisionservices and/or device attributes for one or more of the secondaryclient devices and get them ready for game play so that when a user of aprimary client device selects a handover option, the switching of theclient device is done in a fast and frictionless manner. In someembodiments, device profiling may be initiated in response to occurrenceof a trigger event, such as a user action, a calendar event, a signalfrom a primary client device (e.g., low power signal, etc.), a socialmedia event, in response to a change in one or more device attributes ofthe primary client device, etc.

The secondary client devices may be part of a same network as theprimary client device, Once the secondary client devices have beendiscovered, the primary client device may prompt each of the discoveredsecondary client devices to get profiled, by sending a signal, toinitiate a connection with a server, such as an application server inthe cloud gaming site, and exchange information with the server. Inother embodiments, once the secondary client devices are discovered, theprimary client device may prompt a server of the cloud gaming system toprofile the one or more secondary client devices, by sending a signal toinitiate a connection with each of the discovered secondary clientdevices and exchange information with the secondary client devices. Theinformation that may be exchanged between the secondary client devicesand the server include, for example, session information, quality ofservice information, device capabilities, game being played or availablefor playing, video codec information, audio codec information (whereavailable), speaker configuration, device availability, hardwareattributes, software attributes, etc.

Since each game may have different resource requirements, the deviceattributes that are needed for initiating the streaming session for eachgame may be different. As a result, the information provided in theprofile may be used to evaluate the device capabilities of each of thediscovered secondary client devices to determine if the respective oneof the secondary client devices has sufficient device attributes thatare needed for initiating a streaming session for a game. For example,as part of the evaluation, the network bandwidth of each secondaryclient device may be monitored continuously, periodically, or atpre-defined times. The results from such monitoring may be used toupdate the profiles of the respective secondary client devices so as tomaintain current device attributes of the secondary client devices.Further, each of the primary and secondary client devices may havedifferent types of inputs to provide game interactions. For example,client device A may use a game pad, client B may use a touch screenwhile client C may use a keyboard for providing the game interactionsfor a game that is set up to receive user interactions from such inputs.As a result, the information provided in the profile of the discoveredsecondary client devices, in some embodiments, may be used to identifythe types of inputs that are used by each client device to provide theinput for the game so that when a device handover occurs, an adjustmentto the input method is made. For example, the primary client device maybe using a touch screen input and the selected secondary client devicemay be using a game pad, both of the input methods being acceptable forreceiving interactions for the game. When the device handover occurs,switching of the client device from the primary client device to theselected secondary client device would cause the game to recognizedifferences in the input methods between the two client devices (i.e.,primary client device and the selected secondary client device) andswitch the input method for receiving game interactions for the gamefrom the touch screen input to the game pad input, for example. In someembodiments, the game may not recognize the input method used by theselected secondary client device. In such embodiments, either the cloudgaming system or the selected secondary client device may offer inputemulation, (e.g., an onscreen virtual input method such as a virtualgame pad or virtual keyboard, virtual number pad, virtual game controls,etc.), at a display portion of the secondary client device and the typeof input emulation that is selected for presentation may be based on thetype of client device. Similarly, each client device has its ownresource capabilities and limitations (e.g., screen/display resolution,aspect ratio, audio capabilities, etc.), and these are reflected in theprofile information of the respective client devices. The informationprovided in the device profile may be used to qualify the clientdevices, define input methods for the game, and/or adjust the streamingvideo frames.

Based on the devices profiles of the secondary client devices, a user ofthe primary client device may decide to initiate a device handover to aspecific one of the discovered secondary client devices. The decision toinitiate the device handover may be based on the determination that asecondary client device has device attributes, such as higherresolution, higher frame rate, lower latency, etc., that may provide abetter game playing experience to the user. Alternately, the decision toinitiate the device handover may be based on the fact that the primaryclient device is no longer able to support the game play of the game,due to lack of power in the primary client device, or non-availabilityof one or more device attributes, etc.

As part of device handover, a device handover option along with a listof one or more of the secondary client devices that are pre-qualified toprovide alternate, comparable or better game play experience, isprovided for user selection at the primary client device. User selectioneither at the handover option or at a particular secondary client devicefrom the list, triggers the device handover. As the server has alreadypre-qualified the secondary client devices that are included in thelist, the user selection would result in a faster, efficient,frictionless and seamless device switch so that the server may continuestreaming the video frames for the game to the selected secondary clientdevice. In some embodiments, in addition to switching streaming of videoframes and/or audio frames from the primary client device to theselected secondary client device, the switching may also includeswitching of one or more controllers that are used for providing inputto the game. Oftentimes, a user may use a wireless controller (e.g.,Bluetooth based, or other wireless-technology based) that is associatedwith the primary client device to interact with the game. In suchembodiments, when a device handover is triggered, the switching may alsoinclude a handover of the Bluetooth connection of the wirelesscontroller from the primary client device to the selected secondaryclient device. As part of the device hand over, the primary clientdevice may notify the secondary client device attributes of the wirelesscontroller that is used for providing input to the game. In addition tonotifying the secondary client device, the primary client device maydisconnect the wireless controller. The selected secondary client devicereceives the notification, and may proceed to automatically discover thewireless controller using the device attributes of the wirelesscontroller and form an association with the wireless controller so thatthe selected secondary client device may provide input using the newlyassociated wireless controller. In alternate embodiment, a wirelesscontroller with similar device attributes may already be associated withthe selected secondary client device. In such embodiment, the secondaryclient device may not go through the discovery process for discoveringthe wireless controller or through the association process to associatethe wireless controller with the primary client device. Instead, thewireless controller that is already associated with the selectedsecondary client device may be used for providing input to the game onceit is determined that the wireless controller has necessary deviceattributes for providing input to the game and the game will receive andmap the inputs received from the wireless controller to update gamestate of the game.

The device discovery may, in some embodiments, be initiated by theprimary client device when the primary client device is idle, or may beinitiated during a game play session or periodically or when running adifferent application. The profiling of the secondary client devices areperformed in background to obtain the quality of service and other stateof the secondary client devices so that when and if the user decides tochange devices for game play, the device switch occurs effortlessly asthe device attributes, device settings are pre-established for a quickstream switch. With the general understanding of the invention, specificembodiments will now be described with reference to the variousdrawings.

FIG. 1 illustrates a simplified block diagram of a system used foridentifying options of secondary client devices that can be used fordevice handover of game play, in one embodiment of the invention. Thesystem includes a plurality of client devices 100-1 through 100-n thatare communicatively connected to a server (320), such as an applicationserver, of a data center 310 via a user account of a cloud gaming system300. The server 320 is accessed by the client device 100 over a network200, such as the Internet.

A client device 100 (i.e., any one of 100-1 through 100-n) is anycomputing device that includes a processor, a memory, networkconnections to connect to the network 200, appropriate APIs tocommunicate with a server-side application, and one or more decoders todecode content provided by the server-side application, such as a gameapplication. The processor is capable of executing a client-sideapplication that may interact with a server-side application byconnecting to the network through the network connections and useapplication programming interfaces (APIs) to communicate with or accessthe server-side application. The network connection can be a wired orwireless connection. The client device 100 may be a thin client, ageneral purpose computer, a special purpose computer, a game console, apersonal computer, a lap top computer, a tablet computing device, amobile computing device, a portable gaming device, a cellular phone, asmartphone, a head mounted display, a smart wearable device, a set-topbox, a streaming media interface/device, a smart television or networkeddisplay, or any other computing device that can be used to access anapplication available on a remote server. The network connections andcommunication protocol available at the client device 100 enable theclient device 100 to communicate with the remote server to receivecontent, including streaming video frames of multimedia content, fromthe remote server, such as a server that is part of the cloud gamingsystem 300. The video frames representing game play content streamed bythe remote server have undergone compression using an encoder at theremote server before the video frames are streamed to the client device.The client device 100 may include a decoder to decompress the streamingvideo frames and render content using respective component of the clientdevice 100.

In some embodiments, the amount of processing performed by the clientdevice 100 may vary with respect to input and output processing. Broadlyspeaking, a game or an application is substantially maintained, executedand compressed/encoded on a game server or other application server 320available within a data center 310, with the client device 100 primarilyfunctioning to receive, decode, process and render audio/video data on adisplay of the client device 100. The client device 100 is furtherconfigured to communicate user inputs back to the game server or otherapplication server. The client device 100 may be a standalone computingdevice that is connected to a display for rendering video data. In otherembodiments, the display can be integrated into the client device 100.In one embodiment, the display is a networked display device providing aplatform operating system for applications or “apps” utilizing thenetwork connectivity of the display device. In such an embodiment, theclient device 100 can be defined by an application executed on theplatform provided by the display device's operating system.

During game play session, the client devices 100-1 through 100-n mayactively seek one or more other client devices that are local to it. Theclient device 100 that is used to initiate a game play request isreferred to herein as a “primary” client device and the client devicesthat are local to the primary client device and discovered for devicehandover are referred to as “secondary” client devices. Depending on thelocation of the primary client device 100, the secondary client devicesthat are detected to be local to the primary client device may vary. Forexample, the secondary client devices that are identified to be local toa primary client device 100 in a user's home network may be differentfrom the secondary client devices that are identified in a user friend'shome network, which may be different from the secondary client devicesthat are identified in an office environment, etc.

A server as used in this application may be a remote server, a virtualcomputer, a cloud gaming server, a cloud application server, a remoteapplication server, a digital media server, a server that is used toprovide a storefront of a game developer/game sponsor, a webserver, aterminal server, a console server, or any other type or form of servercomputing device available in a data center that is capable of hostingone or more games or applications (including providing or allocatingprocessing resources for executing the games or applications) that userscan access and interact during cloud gaming. The server may include anencoder to compress the data in the video frames and forward thecompressed video frames in a data stream to the client device 100 usingapplication programming interface (API) calls that follow specific typeof protocol.

For example, a server 320, such as a cloud gaming server in a datacenter 310, executes a video game that is selected for game play by auser, defines a state of the video game from moment to moment, and sendsstreaming video frames (including image, video and audio data) to aprimary client device 100 from where the request for game play of thegame was initiated. A user input module 107 of the primary client device100, in turn, is configured to process user input from a user playingthe video game, and transmit the input data to the server 320. Theserver 320 is configured to process the input data to affect the gamestate of the video game.

The cloud gaming system includes a plurality of data centers 310-1through 310-m. A data center 310, in one embodiment, may include aplurality of servers 320 (e.g., gaming servers), storage systems thatare capable of storing game codes, application codes, user-related andapplication-related data stores and make them readily available toenable handling of varied requests from a plurality of users. The datacenter may also include telecommunication equipment, such as routers,switches, etc., to establish communication connection between the clientdevices 100 and the plurality of servers 320. Each of the plurality ofservers 320 may be equipped with server-side APIs (either distinct orsimilar) to communicate with the corresponding client-side API at aclient device 100 or with server-side APIs associated with third partysocial media providers. In some embodiments, the servers 320 in the datacenter 310 may be configured to execute various types of applications,including gaming applications, and stream the application content (i.e.,video frames generated by the game application) to corresponding clientdevices 100 for rendering. The servers 320 may be configured to performoperations of compression on any data generated or provided by theservers 320 using any number of compression techniques and forward thecompressed data to the client devices 100 using any one of thecommunication and/or transmission protocols. The servers may includeterminal servers, console servers, virtual servers, etc., that aretypically used to perform or execute specific functions, games orapplications. Some examples of functions, games or applicationsperformed by the servers may include database management, filemanagement, mail service, print service, web service, game management,application management, media management, catalog service,communications management, computing service, and proxy management, toname a few. In some embodiments, the servers 320, such as consoleservers, may emulate a game console by executing a game and providingstreaming video frames to the one or more client devices 100 forrendering. In some embodiments, a plurality of servers and/or storagedevices may be provided as rack-mounted servers or storage devices, witheach data center containing rows of server and/or storage racks. Eachserver may be capable of executing a plurality of applications and/orprovide a wide range of services.

The server 320 may also include a device profiler that may be engaged toprofile a client device. The device profiler may be configured toestablish a communication connection with one or more client devices toexchange information with the client devices. The information exchangedis used to identify the device attributes and to generate a profile ofeach client device.

It should be appreciated that the cloud gaming system 300 facilitatessingle-player or multi-player cloud-based gaming from players located atdifferent geo locations by allowing instances of one or more video gamesto be executed by one or more servers 320 (e.g., application or gameservers) within one or more data centers, which can be accessed by theplayers over the network 200. In this manner, execution of the videogame is not dependent on any single player's hardware or networkconductivity, though such will affect the user experience for that givenplayer.

The operations performed using cloud gaming architecture describedherein form technical operations requiring multiple servers and/orexecution platforms to enable quick access to databases and presentationof content, such as game play content, to remotely located clientdevices 100 of users. Cloud gaming can also include the operations ofcompression performed by the cloud gaming servers utilizing any numberof compression techniques. The compression techniques may use standardencoders, which would then allow the standard decoders on client devices100 to access and decode the data. In some embodiments, specializedencoders and decoders may also be implemented in the server and clientdevice, respectively, to allow specialized encoding and decoding. Thedecoded data may be further processed in the client device to identifyimage, video and audio data, which are then presented using appropriatedevice components on the client devices 100, so as to enable game playof the video games.

The management of the video games and distribution can include a numberof data centers, direction servers, quality-of-service testers orengines, direction and redirection to lower latency data centers. Itshould also be understood that these operations and tasks will utilizespecial purpose server computers that are designed for streaming and lowlatency due to the remote execution of games, and the delivery ofstreaming game or application data to the client devices 100.

When a request to access the cloud gaming system 300 is received from aprimary client device 100, a server on an application hosting system,such as a cloud gaming system 300, interacts with user account dataprocessing module 302 to obtain user-related information. The useraccount data processing module 302 queries and receives user accountinformation from a user account database 312, where user profile andother user account information are stored. The user profile and useraccount information are used to identify a user associated with theprimary client device 100 from which the request is initiated, and toauthenticate the user, request. Upon user authentication, the clientrequest is serviced by the server associated with the cloud gamingsystem 300. In one embodiment, the server of the cloud gaming system 300may determine all the games and/or applications that are available forthe user account, including games/applications that were purchased,games/applications that a user of the user account is authorized toview, play, all the games and/or applications that are available forfree. A game title selector module 304 retrieves the game titles for theuser account and returns the game titles in a list for presenting on auser interface at a display of the primary client device 100. In oneembodiment, user selection of any one of the game titles rendered on theclient device, is detected and a signal is sent from the primary clientdevice 100 to the server of the cloud gaming system 300 informing theserver of the user-selection of a game title.

The cloud gaming system 300 hosts a plurality of applications, includinggame applications. In response to receiving the user selection of a gametitle, the server of the cloud gaming system 300 may identify a datacenter 310 to service the request of game play of the selected gametitle and forward the request to the data center 310 for servicing therequest. The data center 310 may be selected based on a geo location ofthe primary client device 100, for example. Selecting a data center 310to service the request based on geo location may have its owndisadvantages. For example, the data center 310 at the geo location mayexperience network congestion due to high demand on its resources.Alternately, the data center 310 at the geo location may not be able toservice the request for game play from a particular primary clientdevice due to lack of peer-to-peer agreements between the networkprovider that services the primary client device and the networkprovider that is associated with the operation of the data center. Theaforementioned disadvantages are just examples and that otherdisadvantages may also exist. To overcome such disadvantages, in anotherexample, the data center 310 may be selected based on the results ofnetwork performance tests, such as bandwidth/latency tests, etc. Forexample, when a primary client device is used to initiate a game titleselection for game play, the server of the cloud gaming system 300 maydetermine a geo location associated with the primary client device andidentify one or more data centers 310 to service the request. Whenassigning a data center, the cloud gaming system 300 may realize thatservers within a data center 310 that is in the geo location of theprimary client device may not have sufficient resources to service therequest. As a result, the cloud gaming system 300 may identify a datacenter 310 that is outside the geo location in which the primary clientdevice is based, to service the request. The data center outside the geolocation may be identified over other data centers in the geo locationassociated with primary client device, based on the bandwidth test, forexample.

As mentioned earlier, the data center 310 may include a plurality ofservers 320, with each server 320 executing or set up to execute one ormore game titles that are available at the data center 310. A game titledata store 314 in each data center 310 provides the list of game titlesthat are available at the data center 310 and the necessary game codefor executing the games identified by the game titles, at one or moreservers 320 in the data center 310.

A server 320 in a data center 310 that is identified for servicing thegame play request of a game establishes a game play session by executingan instance of the game, generates video frames for the game, andstreams the video frames to the primary client device 100 for rendering.The server receives user interactions provided at the primary clientdevice 100 and uses it to affect an outcome of the game and to updatethe game state. The client device may be used to identify other clientdevices that are local to it. The identified other client devices areprovided as options for a device handover.

FIG. 2 illustrates the various modules of a server-side application 321executing on a server 320 within a data center 310 of a cloud gamingsystem 300 that is used to identify options of secondary client devicesfor a device handover during game play, in one embodiment. Theserver-side application 321, in one embodiment illustrated in FIG. 2 ,may include modules of a game application 321-a integrated with modulesof a handover application module 355. The modules of the gameapplication 321-a, are used to execute the game and provide streamingvideo frames of the game during game play to a client device, such as aprimary client device 100, in some embodiments. In such embodiments, thevideo frames are encoded prior to transmitting the same to the primaryclient device. In some embodiments, application programming interfaces(APIs) that are part of the game application may be used fortransmitting the video frames. The APIs may include the logic to capturethe video frames generated by the game application, and use encodingtechnology available within to encode the video frames and transmit theencoded video frames to a client device, such as a primary client device100. It should be noted that the various embodiments discussed hereinfor processing the video frames may also be extended to processing ofthe audio frames.

In some embodiments, the encoding of the video frames may be based on‘regions of interest’ provided by the game. Usually, when a frame ofgame data is to be encoded, the encoder has to adhere to a ‘bitsbudget’. Bits budget defines how bits are to be used for encoding theframe so as to provide optimal data content. Relying on the bits budget,the encoder attempts to assign more bits to certain areas of the frame(e.g., areas of the frame that have dense content) while assigning lessbits to other areas of the frame (e.g., areas that have less content).In order to assist the encoder to optimally encode the video frames, thegame provides information related to the regions of interest so that theencoder can identify areas that deserve more detail (i.e., areas withtext, intense game scene, such as grass, leaves, forests, etc., intenseactions, etc.) and areas that deserve less detail (i.e., less noticeableareas of the frame). Equipped with this level of detail from the gameand the bits budget, the encoder assigns the bits for the video frame,so as to provide optimal frame content for viewing.

In alternate embodiments, the streaming video frames of the gamegenerated during game play are forwarded to an operating system of theserver 320 or to graphical processing unit (GPU) drivers that interfacewith the game application 321-a and/or the operating system. The GPUsmay process the video frames and provide it to the operating system ofthe server 320. The operating system of the server 320 may receive theprocessed video frames and, in the background, perform some form ofencoding before transmitting to the primary client device 100. In someembodiments, the operating system of the server 320 within the cloudgaming system 300 may engage an encoder available within the cloudgaming system 300 to perform the encoding of the video frames prior totransmitting the same to the primary client device 100 for rendering. Asin the previous embodiments, the encoder may receive detailedinformation related to regions of interest from the game and encode thevideo frames by assigning the bits of data that make up the video framesin accordance to the detailed information. The embodiments discussedherein with reference to processing of video frames may also be extendedto the audio frames. The operating system may monitor the networkconditions and responsively adjust the frame rate, bit rate, etc., ofthe video frames and/or the audio frames prior to forwarding the videoframes and/or audio frames to the client device.

In alternate embodiments, an external application that is outside of thecloud gaming system 300 may capture High-Definition Multimedia Interface(HDMI) video signals representing the streaming video frames that arebeing transmitted by the operating system of the cloud gaming system300, and may engage an external encoder to encode the video signals. Theexternal application then handles the transmission of the encoded videosignals to the client device. As part of transmission, the externalapplication may monitor the network conditions and adjust the quality(i.e., frame rate, bit rate, etc.) of the video frames that is streamedto the primary client device 100.

In some embodiments, based on the network conditions, the operatingsystem or the external application may automatically trigger a devicehandover. As part of the triggering, the operating system or theexternal application may discover other secondary client devices thatare in the vicinity of the primary client device, signal the discoveredsecondary client devices to get profiled by the server, and forward thesecondary client device information to the handover application module355 for use during a device handover.

The modules of the handover application module 355 are used to identifysecondary client devices that are local to the primary client device, oruse the secondary client device information provided by the operatingsystem or an external application, profile the secondary client devicesand provide them as options for a device handover. In other embodiments,the server-side application 321 may include a game application 321-athat is separate from the handover application module 355. In suchembodiments, the game application 321-a and the handover applicationmodule 355 may be separately executed on the same server or the gameapplication 321-a may be executed on a first server and the handoverapplication module 355 may be executed on a second server in the datacenter, with the first server and the second server in communicationconnection with one another to exchange data, such as user selection,game state, pause/resume requests, etc. In some embodiments, the gameapplication 321-a that is generating the streaming video frames and/oraudio frames may have the capability to monitor the network connectionto the primary client device and share this information with thehandover application module 355. Alternately, the network connectionquality information may be provided to the handover application module355 by the operating system of the server or an external application.

The handover application module 355 may use the network connectioninformation of the primary client device and the profiles generated forthe secondary client devices to determine if one or more secondaryclient devices that are local to the primary client device have betternetwork connection quality than what is available at the primary clientdevice. When the handover application module 355 identifies one or moresecondary client devices, the handover application module 355 maygenerate an informational message with an option to switch to alternatesecondary client devices for improving game play experience. Theinformational message along with a list of alternate secondary clientdevices may be forwarded to the primary client device for rendering onthe client-side for user selection, during a device handover. In someembodiments, the informational message may be provided as an overlay forrendering on top of game scene that is rendering on a display portion ofthe primary client device. Alternately, the informational message may berendered server side by the game application 321-a or by a server sideprocess, or by a user interface of the server side game console, etc. Insome embodiments, the user interface of the server side game console ishidden but renders notifications in response to trigger actions, such asreceiving a new email, receiving a new message, detecting a user or afriend of a user coming online, etc., wherein the trigger actions may beextended to also include receiving informational message.

The function or role of the various modules will be explained withreference to the integrated server-side application model illustrated inFIG. 2 but can be extended to other server-side application modeldiscussed above. As mentioned earlier, the server-side application 321includes modules of the game application 321-a integrated with modulesof a handover application module 355. The plurality of modules of gameapplication 321-a enables the server 320 to identify device identifierof a primary client device 100, identify and execute an instance of agame requested by the primary client device 100, and to providestreaming video frames for forwarding to the primary client device forrendering. The plurality of modules of the handover application module355 enables the server to discover and qualify one or more secondaryclient devices and to provide a list of qualified secondary clientdevices as options for a device handover.

Some of the modules within the game application 321-a portion of theserver-side application 321 that are used to provide game related datato a client device 100 include a primary client device identifier 322, agame execution engine 324, and an encoder 328, to name a few. Themodules in the handover application module 355 include modules foreffectuating a device handover and modules for profiling one or moresecondary client devices. The modules for effectuating device handoverinclude a handover manager 330, a resume manager 332, and a pausemanager 334, to name a few. The module for profiling one or moresecondary client devices includes a client device profiler 340.

In response to a user selection of a particular game title for gameplay, the cloud gaming system 300 identifies a data center 310 that isin or proximal to a geo location of the primary client device, and isconfigured to service the game play request. The cloud gaming system 300forwards the game play request to a server at the identified datacenter, which identifies the game title from the game title data store314 that is within the identified data center.

As part of servicing the request, the server 320 identifies the clientdevice 100 that initiated the game play request using a primary clientdevice identifier 322 and designates the client device 100 as a primaryclient device 100. The identity of the primary client device 100 is usedfor forwarding the streaming content. The server 320 then establishes agame play session by providing necessary resources for game execution(e.g., storage, processor, etc.) and directing a game execution engine324 of the game application 321-a to retrieve the game code of the gameand instantiate the game by executing the game code for the game on theserver 320. As part of instantiating the game, the game execution engine324 generates video frames 326 for the game. The video frames 326 arecompressed in accordance to a communication protocol defined for theprimary client device 100, using an encoder 328. The encoded game datais streamed to the primary client device 100 for rendering. In someembodiments, the encoder 328 may be part of the game application 321-a.An Application Programming Interface (API) of the game application 321-ais used to stream the encoded video frames to the primary client device100. In some embodiments, the APIs that are part of the game application321-a may include encoding logic to encode the video frames and forwardthe encoded video frames to the primary client device. In alternateembodiments, the video frames generated at the game execution engine 324are captured by an operating system of the server 320 and processed. Asnoted previously, in some instances, the operating system may encode thevideo frames, monitor the network conditions and adjust quality of thevideo frames before the encoded video frames are streamed to the primaryclient device 100. Alternately, the operating system may process thevideo frames and forward the same as HDMI video signals. An externalapplication that is outside of the cloud gaming system may capture theHDMI video signals, encode the video frames captured in the HDMI videosignals using an external encoder, monitor the network conditions andadjust the resolution of the images in the video frames and/ortransmission rate prior to transmitting the video frames to the primaryclient device 100 through one or more APIs available within the externalapplication. It should be noted that the process used for capturing,monitoring network conditions, encoding and transmitting the videoframes may be extended to audio data, as well. User interaction at theprimary client device 100 is received by the server and used to adjustgame outcome at the game execution engine 324.

During the game play session, the primary client device 100 may activelyseek out other client devices (also referred to herein as secondaryclient devices) that are identified to be local to the primary clientdevice 100. In one embodiment, upon detecting a secondary client device,the primary client device 100 may establish a network connection withthe secondary client device and send a signal to the secondary clientdevice to communicate with the server 320 and get profiled. It may beadvantageous to have the secondary client device establish the networkconnection as opposed to the server 320 due to presence of securitydevices, such as firewalls, at the server 320. Establishing a networkconnection through the secondary client device may be enabled throughone or more background agents that are available on secondary clientdevices. For example, usually the secondary client device is alwaysconnected to a social media server, an email server, etc., and receivesa social media notification or email notification, etc. The primaryclient device may access the secondary client device via this networkconnection and send a signal to the secondary client device to profileitself. The signal may include a link to the server, an IP address ofthe server, etc., to enable the secondary client device to identify andestablish a communication link with the appropriate server in order toinitiate a profile request 105. In alternate embodiment, upon detectingthe secondary client device, the primary client device 100 may send asignal with a profile request 105 to the server 320 requesting theserver to establish a communication link with the secondary clientdevice and generate a profile of the secondary client device detected bythe primary client device. In this embodiment, the signal from theprimary client device 100 may include the device identifier of thesecondary client device.

The server 320, in response to the signal from the primary or thesecondary client device, may establish the communication connection withthe secondary client device. The communication connection allows aclient device profiler 340 of the server-side application 321 to requestand receive information, such as network bandwidth information, qualityof service information, device capabilities including memory, processorspeed, etc., device availability, game/application availability, etc.,from the secondary client device. The information exchanged is used togenerate a profile of the secondary client device. The profileidentifies the device attributes of the secondary client device frominformation exchanged with the server. As additional secondary clientdevices are discovered by the primary client device, the additionalsecondary client devices are also profiled by the client device profiler340, in response to the profile request. The profile information for thedifferent secondary client devices is used by the client device profiler340 to generate a list of secondary client devices 342 that can be usedfor device handover.

In an alternate embodiment, instead of the primary client deviceactively seeking secondary client devices, different components of theserver may actively seek secondary client devices that are local to theprimary client device that is actively involved in the game play sessionof a game available on the cloud gaming system. For example, the gameapplication 321-a or an API in the game application 321-a, API withinthe game server that processes the video frames, or an externalapplication within the cloud gaming system that processes the videoframes may actively be involved in discovering secondary client devicesfor device handover. In such embodiments, the server may determine geolocation of the primary client device using signals generated by theprimary client device or based on information obtained from social mediaposts that a user interacts with using the primary client device (eitherbefore or during the gameplay session) or from tools embedded in theprimary client device or from applications implemented in the primaryclient device, etc. Using the geo location of the primary client device,the server may seek to discover one or more secondary client devicesthat are identified to be local to the primary client device using oneor more device discovery protocols implemented in the various clientdevices, for example. In some embodiments, the secondary client devicesare registered with the cloud gaming system and are either activelyinteracting with the cloud gaming system or are dormant. In otherembodiments, the secondary client devices may be actively interactingwith other content provider system but may be set up for cloud gaming onthe cloud gaming system.

Once the secondary client devices are identified, the client deviceprofiler 340 may generate profiles of the primary client device and thesecondary client devices and such profiling may be done automatically asand when a secondary client device is detected by the server or inresponse to a trigger event (e.g., low-battery power signal, increasedlatency, a calendar event, etc.). Profiles of the primary and thesecondary client devices may be used by the client device profiler 340to determine if any one or more of the secondary client devices hasdevice attributes that can enhance the game play experience of the gamecurrently being played, for the user. When it is determined from thegenerated device profiles that one or more secondary client devicesinclude device attributes that can provide better game play experience(enhanced resolution, improved frame rate, lower latency, etc.) thanwhat is being provided by the primary client device, the client deviceprofiler 340 identifies the select ones of the secondary client devicesand generates a list of secondary client devices.

The list of secondary client devices 342 is forwarded to the handovermanager 330. The handover manager 330 is configured to query the gameexecution engine 324 to determine the device attribute requirements forthe game. Each game may have its own device attribute requirements. Forexample, a graphic intense game may require a higher network bandwidthwhile a computation intense game may require a faster processor. As aresult, the device attribute requirements may be specific to the gameand the secondary client devices are being identified based on thedevice attribute requirements of the game. The handover manager 330 usesthe device attribute requirements for the game to qualify the secondaryclient devices provided in the list 342. Additionally, in someembodiments, the handover manager 330 may identify one or more deviceattributes or services that need to be provisioned for a secondaryclient device in order for the secondary client device to be qualifiedfor game play of the game. The handover manager 330 may then proceed toprovision the required device attributes or services in order to qualifythe secondary client device for game play. The list 342 is refined bythe handover manager 330 to include only those secondary client devicesthat are qualified for game play of the game. The secondary clientdevices identified in the refined list may be a subset of the secondaryclient devices provided by the client device profiler 340.

It should be noted that the subset of qualified secondary client devicesmay include one secondary client device from the list 342 provided bythe client device profiler 340, some of the secondary client devicesfrom the list, or all of the secondary client devices in the list. Thehandover manager 330 forwards this refined list of qualified secondaryclient devices to the primary client device 100 with a handover option,for user selection, as shown by a directional arrow represented bybubble ‘a’.

The handover option with the list of qualified secondary client devicesis rendered on the display of the primary client device. In oneembodiment, the list of qualified secondary client devices may berendered as a client-side overlay style menu over the game scenes of thegame and options may be provided at the primary client device forselecting any one of the qualified secondary client devices. In anotherembodiment, the list of qualified secondary client devices may berendered on the server-side. In such embodiments, the list of qualifiedsecondary client devices may be part of the video stream that isforwarded to the client device for rendering. User selection of thehandover option and/or a qualified secondary client device is passedover to the handover manager 330, over the network 200, as shown by adirectional arrow represented by bubble ‘b’. In one embodiment, inresponse to receiving user selection from the primary client device, thehandover manager 330 may then proceed to instruct or send a command orsignal to a pause manager 334 to pause the game play of the game, asillustrated by directional arrow represented by bubble ‘c’. The pausecommand or instruction will include the identifier of the primary clientdevice as well as the secondary client device identified in the userselection. In response to the pause command, the pause manager 334signals the game execution engine 324 to pause the game play. In analternate embodiment, the handover manager 330 may send a signaldirectly to the game execution engine 324 to pause the game play.Irrespective of which module provides the pause signal, the gameexecution engine 324, in response to the pause signal, would interactwith a state data manager 338 to determine a game state of the game at atime when the pause request was received, in one embodiment. The statedata manager 338 is configured to analyze game play of the game todetermine the game state of the game at the time the pause signal wasreceived, and to identify a restart point from where the game play maybe resumed. The game state of the game and the restart point may bemaintained by the state data manager 338 for the duration of the gameplay session. In an alternate embodiment, the state data manager 338 maywrite save data of the game that includes progression made in the gameduring game play up till the point when the pause request was received.This save data may be used to re-create the game play of the game whenthe game play is resumed. Further, as part of pausing the game play, thegame execution engine 324 suspends streaming video frames to the primaryclient device identified in the pause signal.

In addition to providing a pause signal, the handover manager 330 mayprovide a resume option to the secondary client device identified in thepause signal to allow the secondary client device to resume game play ofthe game. User selection of the resume option from the secondary clientdevice causes a resume request signal to be forwarded from the primaryclient device 100 to the handover manager 330 on the server.

The handover manager 330 may verify the resume request is from theselected secondary client device. Upon verification, the handovermanager 330 may forward the resume request to a resume manager 332 forprocessing. The resume manager 332 may interact with the game executionengine 324 to determine the game state of the game. The game executionengine 324 receives the request from the resume manager 332 and queriesthe state data manager 338 to obtain the game state of the game and gamerestart point for the game. The resume manager 332 may then instruct thegame execution engine 324 to resume game play of the game and stream thevideo frames from a re-start point identified for the game, to theselected secondary client device. As part of resuming the game, theresume request may include capabilities of the selected secondary clientdevice, such as screen display resolution, input controls available,audio capabilities, etc. The capabilities of the selected secondaryclient device provided with the resume request may be used to adjust thestream of video frames so that the video frames can be rendered on theselected secondary client device. Further, the input methods used by thesecondary client device are used to adjust the game play so that thegame can recognize the interactions provided using the input methodsavailable at the secondary client device. The video frames 326 arecompressed in accordance to the communication protocol that is definedfor the secondary client device prior to streaming to the secondaryclient device. The compression technique used for compressing the streamof video frames sent to the primary client device and the secondaryclient device may be same or different depending on the type of theprimary and secondary client devices chosen for game play.

Meanwhile, the handover manager 330 uses the secondary client deviceidentifier that was provided in the resume request to update the primaryclient device that will be interacting with the game execution engine324. The handover manager 330 instructs an update primary client devicemodule 336, which in turn sends a signal to the primary client deviceidentifier module 322 to update the client device identifier with thesecondary client device identifier, as illustrated by bubble ‘d’, sothat the game execution engine can identify the client device to beginstreaming the encoded video frames. As part of updating, the primaryclient device identifier module 322 replaces the primary client deviceidentifier with the secondary client device identifier and designatesthe secondary client device as the primary client device 100.

In some embodiments, the handover option may allow selection of morethan one secondary client device from the list of qualified secondaryclient devices for device handover. In such embodiments, user selectionof the resume request option from a specific secondary client devicewill determine which one of the secondary client devices to stream thevideo frames from the resumed game play. Selection of more than onesecondary client device may be to provide the user additional options ofdifferent qualified devices, so that the user can select a device basedon the device availability, network bandwidth, communication signalstrength, available frame rate, quality of service, etc. For example,two secondary client devices may be identified to have similar deviceattributes but one may not be available for the entire duration of thegame session from the time when a handover request is received, or mayhave low communication signal strength or power, etc. In such cases,providing the resume request option on both the secondary client devicesmay help the user in switching to the appropriate secondary clientdevice for playing the game, during a device handover.

FIG. 3 illustrates various modules within a primary client device thatmay be used to identify options of secondary client devices and forselecting a secondary client device for device handover, in oneembodiment of the invention. The primary client device is a computingdevice that includes components, such as a processor 104 for processingrequests and data received from a server, such as a cloud gaming server320, one or more memory units for storing data, network components andcommunication protocols for connecting to the internet, and forinteracting with other devices, such as servers, consoles, other clientdevices, controllers, network devices, etc. The primary client device100 may be a laptop computer, a desktop computer, a mobile phone, atablet computer device, a head mounted display, etc. A user login module102 is used to provide a user interface for rendering on a display ofthe primary client device 100 and is used for providing userauthentication information for accessing a cloud gaming service, forexample. The user interface may also be used to provide user selectionsand user interactions during game play. The processor 104 includes astream processor 106 that is used to receive and process the streamingcontent received from a server, such as a cloud gaming server 320 orother content provider over a network 200. The stream processor 106includes necessary logic to verify that the streaming content receivedfrom the server adheres to the communication protocol defined for theprimary client device 100, and to process the content. The processor 104may engage a decoder 108 that is available to the processor todecompress the encoded streaming data provided by the server 320. Thedecompressed data is forwarded to a data handler 110 within theprocessor 104 to analyze the data in order to identify the various datacomponents and to separate the various data components of content. Thestreaming content (i.e., streaming video frame data) from the server 320may include video frame data, audio data, haptic data, images, etc. Thedata handler 110 separates the various data components provided in thestreaming content and forwards the different data components separatelyto the appropriate device components on the primary client device 100,for rendering. For example, the audio data component 110-a may beforwarded to a speaker for rendering, the video 110-b and image datacomponent may be forwarded to the display for rendering, etc. Userinput, such as user selection, game interactions, etc., provided at theclient device and captured by a user input module 107, is processed bythe processor 104 and forwarded to the server 320, in response to thestreaming content, so as to drive an outcome of a game, for example.

The primary client device 100 also includes a switching module 120 thatis used to actively seek and detect secondary client devices that are ina vicinity of the primary client device 100 so as to present a handoveroption for a device handover for playing a game. The switching module120 includes a plurality of sub-modules, such as a discovery agent 130,a device profile instructor 140 and a handover logic 150, to name a few.The discovery agent 130 is used to discover other client devices thatare identified to be local to the primary client device. The discoveryagent 130 uses various signals emitted by the client devices and/orprotocols to seek other client devices. The discovery agent 130 will bedescribed in more detail with reference to FIG. 4 .

The device profile instructor 140 is used to signal to the secondaryclient devices that were discovered by the discovery agent 130 toestablish a communication connection with a server of the cloud gamingsystem and get profiled, in some embodiments. In other embodiments, thedevice profile instructor 140 may signal the server of the cloud gamingsystem to establish a communication connection with the discoveredsecondary client devices and to profile each one of the secondary clientdevices. More details of the role of the device profile instructor 140will be discussed with reference to FIG. 5 .

The handover logic 150 is used to provide appropriate user interface torender a handover option and a list of secondary client devices that arequalified for game play of a game, in order to successfully perform adevice handover. More details of the function of the handover logic 150will be discussed with reference to FIGS. 6A, 6B, 6C-1 and 6C-2 .

FIG. 4 illustrates various sub-modules of a discovery agent 130available on a client device 100 for seeking and discovering otherclient devices. The sub-modules of the discovery agent 130 may bebroadly classified into two main sub-modules—a device location detector132 and a service discovery module 134. The device location detector 132is configured to identify a geo location of the primary client device byusing signals emitted by the primary client device or by obtaining datafrom components within the primary client device or devices external tothe primary client device. In a local network, such as a home network,various client devices, such as tablets, smart televisions, smartwatches, mobile phones, cable television boxes, game consoles, etc., areall connected to the home network through an Ethernet connection (i.e.,wired connection), a Wi-Fi (i.e., wireless) connection or a combinationof both. As a result, the geo location of the primary client device maybe determined, in one embodiment, by evaluating a Wi-Fi signal strengthexhibited by the primary client device to one or more network accesspoints within the network, and such evaluation may be performed using aWi-Fi signal detector 132-a. In an alternate embodiment, a local routerdevices detector 132-b may query a local router device to obtain a listof client devices that are connected to the network through the routerdevice. In still another embodiment, a global positioning service (GPS)tool 132-c implemented in the primary client device 100 may be used topinpoint the geo location of the primary client device. In furtherembodiments, the geo location of the primary client device may bedetermined by interrogating signals between the primary client deviceand one or more cell towers, using a cell tower signal detector 132-d.This mode of detecting geo location of the primary client device may beuseful when the primary client device is set up to interact with celltowers, such as a mobile phone. In some embodiments, a combination ofthe various device location detectors 132 may be used to determine thegeo location of the primary client device 100 of the user.

Once the geo location of the primary client device is determined, one ormore service discovery modules 134 may be used to actively seek anddiscover other client devices that are in the same network environmentas the primary client device 100. For example, a Domain Name System(DNS) based protocol, such as a multicast DNS service discovery protocol134-a, which is implemented across various devices, such as tabletcomputers, laptop computers, printers, etc., may be used to discoversecondary client devices that are available within the network in whicha primary client device operates. Alternately, a service locationprotocol 134-b may be used to detect secondary client devices in a localarea network within which the primary client device operates. Otherservice discovery protocols may include a blue-tooth service discoveryprotocol 134-c for detecting Bluetooth enabled secondary client devicesin a local area network of a primary client device, using wirelesstechnology. The list of service discovery protocols provided herein aremere examples and that other service discovery protocols may be used toautomatically seek and discover secondary client devices within a localnetwork in which the primary client device operates. The varioussecondary client devices discovered using the different protocols areforwarded to the server 320 over network 200 using a wireless connection138, for example. The server may receive the device identifiers of thediscovered secondary client devices and establish a communicationconnection to exchange information that is used to identify deviceattributes and to profile the respective secondary client devices.Alternately, a signal may be sent to the various secondary clientdevices with a request to establish a communication connection with theserver 320 and to exchange information with the server. The informationexchanged with the server is used to generate the profile of thesecondary client devices. The profiles generated for the secondaryclient devices are used to qualify the secondary client devices so thatappropriate secondary client devices may be identified and provided asoptions for a device handover.

FIG. 5 illustrates an example client device profiler 340 that is used toqualify the secondary client devices that were discovered to be local tothe primary client device, in order to provide options of qualifiedsecondary client devices for a device handover, in one embodiment. Theclient device profiler 340 (of FIG. 2 ) within the server 320 receivessignal(s) to profile one or more secondary client devices discovered bythe primary client device 100 using various device discovery protocols,and such signals may be received from either a device profile instructor140 of a client device or from one or more secondary client devices. Inresponse to the signal(s), the client device profiler 340 establishescommunication connection and begins exchanging information with therespective secondary client devices. Various sub-modules within theclient device profiler 340 may be used for obtaining the informationfrom the secondary client devices. For example, a quality of service(QOS) evaluator 341 may be used to obtain QOS information from thesecondary client device. The QOS information may include networkbandwidth information, frame rate, streaming capability, and othernetwork related information, such as ping time, packet loss, link type(i.e., network connections using WiFi, Ethernet, Long Term Evolution(LTE) technology, etc.) etc., that affect game play experience of auser. In some embodiments, the QOS information may be used to organizethe secondary client devices discovered by the primary client device. Inother embodiments, the information exchanged between the secondaryclient devices and the client device profiler 340 may be used toorganize the secondary client devices. For example, the deviceattributes may be ranked in accordance to the device attributerequirements of a game. A game that is process intensive may rankprocessor speed higher than other device attributes and a game that isgraphic intensive may rank network bandwidth higher than other deviceattributes. The QOS evaluator 341 may rank the device attributes inaccordance to the requirements of a game and use this ranking of thedevice attributes in identifying and qualifying secondary client devicesdiscovered by a primary client device, for a device handover. In someembodiments, the qualified list of secondary client devices areorganized in a list in accordance to level of matching of deviceattributes of the secondary client devices to the ranked deviceattributes.

A device specification discovery 349 may be used to determine thehardware and software specification of the secondary client deviceincluding, but not limited to, type and version of operating system,type of processor, processor speed, type and amount of memory, etc. Thedevice specification may be used for categorizing the secondary clientdevice and for packaging the streaming video frames for forwarding tothe secondary client device.

A device availability module 343 may be used to determine theavailability of each of the secondary client devices discovered by theprimary client device. Device availability may drive the selection ofqualified secondary client devices for providing as options for a devicehandover. Some or all of the secondary client devices may be connectedto the local network at the time of discovery. One or more of thesecondary client devices may be committed to provide services for anevent, such as a calendar event, a user action event, etc. When theevent occurs, the resources of the one or more secondary client devicesthat were committed to provide service for the event may be engagedduring the event and those secondary client devices are shown to be in a‘busy’ state. In order to determine if a particular secondary clientdevice is available during a device handover, the device availabilitymodule 343 is configured to query an event mechanism (not shown) todetermine the various events for which the resources of a secondaryclient device may be used or is committed. The event mechanism, in oneembodiment, may be configured to query and obtain information related tothe events from various scheduling sources in order to identifysecondary client devices for a device handover during a game session.

It should be noted that the device availability information can be usedto determine which ones of the secondary client devices are availableduring a device handover at a time when a handover option is receivedfrom the primary client device. Alternately, the device availabilityinformation may be used to determine which ones of the secondary clientdevices are scheduled to be busy for a game play session that isscheduled for some time in the future. An interface, such as a networkinterface 347, may be used to provide interface logic and/or APIs toquery the different scheduling applications, such as a calendarapplication 345, a social media application 346, an email application,etc., to determine if any event has been scheduled for a current time ora time when a game play session is being scheduled for which resourcesof a particular secondary client device has been committed or isrequired. For example, if a user committed to play an online multiplayergame through a social media post for the current time, the eventmechanism may be able to extract such information by querying the socialmedia application 346, which relies on one or more social graphs 346-ato identify social contacts. Identity of the social contacts may be usedto determine which social posts/feeds to monitor or query to obtain theuser's social commitment of playing the game. Similarly, if a user has acalendar entry for playing the online multiplayer game on every Fridayat 8:00 p.m., the event mechanism may be able to extract suchinformation from the calendar application 345 or from an emailapplication or a calendar option in a social media or other schedulingsources. The device availability module 343 analyzes the various eventsthat a particular secondary client device has been committed to in orderto determine which ones of the identified secondary client devices areavailable for including in a device handover option. Schedulinginformation obtained from the device availability module 343 is used foridentifying and qualifying the secondary client devices provided asoptions with a handover option. Of course, the secondary client devicesthat are identified have device attributes to provide optimal game playexperience for the user.

A game/apps discovery module 344 is used to determine the variousapplications and/or games that are available at the data center that isassigned for the game play session of a game. Each data center, as shownin FIG. 1 , includes a plurality of games that are available or are setup for playing. The plurality of servers provides the necessaryresources for playing the games available in each data center. Thegame/apps discovery module 344 is configured to determine the list ofgames that are available and to determine if any game needs to beprovisioned at the data center to enable game play. If a particular gameis not available at the data center, the cloud gaming system 300 mayprovision the game at the data center so that when and if the game isselected for game play, an instance of the game may be retrieved fromthe games title data store 314, executed on a server of the data centerand game content streamed to the primary and the secondary clientdevices. Similarly, if a secondary client device does not have one ormore device attributes or if the device attributes are not to the levelrequired for game play of a game, the game/apps discovery module 344 mayidentify such attributes and the cloud gaming system 300 may be used toprovision those attributes to qualify the secondary client device forplaying the game. The client device profiler 340 uses the informationgathered by the various sub-modules to generate device profile of thevarious secondary client devices that were discovered to be local to theprimary client device.

FIG. 6A illustrates an example device profile that is generated by theclient device profiler 340 for all the secondary client devices that areidentified to be local to the primary client device. The device profileidentifies the secondary client devices using device identifiers, andlists out the various device attributes of the secondary client devices.For simplicity sake, the device identifiers of the secondary clientdevices are listed in numerical order whereas in reality the deviceidentifiers are unique identifiers that can be used to distinctlyidentify each client device. The number of secondary client devices andspecific device attributes has been provided as examples and should notbe considered limiting. Fewer or greater number of device attributes maybe identified in the device profile generated by the client deviceprofiler 340.

The profile generated by the client device profiler 340 may be used bythe handover manager 330 at the server 320 to qualify the secondaryclient devices and to identify specific ones of the secondary clientdevices that are qualified for providing with a handover option for adevice handover, during game play of the game. The qualified secondaryclient devices are returned with a handover option to the primary clientdevice 100 for rendering and user selection.

FIGS. 6B, 6C-1 and 6C-2 show different examples of user interface thatare rendered on a display of the primary client device. A handover logic150 at the primary client device 100 receives the handover option andthe list of qualified secondary client devices from the handover manager330 at the server 320 and presents the handover option with the list ofqualified secondary client devices in a user interface 153 for userselection. The user interface identifies the various options that areavailable to a user (i.e., “Pause” or “Resume”). Further, the userinterface provides the device identifier of the primary client device(D1), as shown in box 155, and a list of qualified secondary clientdevices under “select other device” option 157. FIG. 6B illustrates theuser interface 153 rendered on a display of the primary client device,in one embodiment. The list of secondary client devices provided underoption 157 for user selection is a subset of secondary client devices(illustrated in FIG. 6A) that were provided by the client deviceprofiler 340 and, in one embodiment, is organized in accordance to thedevice attributes of the respective secondary client devices as comparedwith the device attribute requirements of the game.

A user selection of a “Pause” option, as shown by the black cursor,would result in a signal being sent to the handover manager 330 at theserver 320 to pause game play of the game. The handover manager 330, inresponse, would result in sending instructions to the game executionengine 324 to suspend streaming video frames to the primary clientdevice. The handover manager 330 further sends a signal to the pausemanager 334 to save a game state of the game play to enable restart ofthe game play of the game from the point where it was paused. The gamestate of the game play is saved by a state data manager 338. A userselection of a secondary client device, as illustrated by the graydotted cursor, would cause the secondary client device to be identifiedas the handover device to which the game play has to be streamed, if andwhen a user decides to resume game play at the selected secondary clientdevice. The user selection of pause option would also result in a“Resume” option to be forwarded to the selected secondary client device.User selection of the resume option at the secondary client device isforwarded to the handover manager 330 at the server 320. The handovermanager 330, in turn, would instruct the resume manager 332 to resumegame play of the game. The resume manager 332 would instruct the gameexecution engine to retrieve the game state from the state data manager338 and restart the game from a restart point that is saved in the statedata manager 338. The handover manager 330 sends a signal to update theprimary client device identifier to reflect the selected secondaryclient device identifier, so that the game execution engine 324 can usethe device identifier of the selected secondary client device to streamthe video frames for the restarted game.

FIGS. 6C-1 and 6C-2 provide an alternate way of selecting a secondaryclient device as the handover device, in one embodiment of theinvention. As illustrated in FIG. 6C-1 , a user may select one of thesecondary client devices from select device option 157, as illustratedby the black cursor, for a device handover, instead of selecting the“Pause” option. The user selection of the secondary client device wouldautomatically cause the selected device to be highlighted in the selectdevice option 157 and the pause option to get activated, as illustratedby the highlighted “Pause” option in FIG. 6C-2 . The process of pausingthe game play of the game, saving the game state and providing a“Resume” option, in response to the user selection of secondary clientdevice, is similar to what was discussed with reference to FIG. 6B. Gameplay may resume if and when the user selects the resume option from thesecondary client device.

The various embodiments discussed herein allows a primary client deviceto use various protocols and signal detectors implemented in each of theclient devices to actively seek and discover secondary client devicesthat are local to the primary client device. The protocols andtechnology available within each client device, in some embodiments,enables an actively engaged client (i.e., primary client device) that isreceiving streaming content from a content provider (e.g., gamingserver) to discover other dormant client devices that are eligible toreceive streaming content, other idling client devices (e.g., clientdevices with idling socket waiting for messages) that could be woken upto become a client that can receive streaming content, or a standbyclient that is eligible for receiving streaming content but is currentlyperforming limited work like recording a show, or downloading somecontent. In some embodiments, the actively engaged client may promptother client devices to exchange information (e.g., session information,QOS information, device capabilities, game being played, video codecinformation, etc.) and initiate preparation work for a streamingsession. This could include monitoring network bandwidth. If theactively engaged client is a table computer that is running low onbattery power, a trigger event (i.e., low batter trigger event) mayoccur prompting a user interface to appear on the tablet computersuggesting to the user to switch or move game play to another clientdevice. The suggestion may include other client devices that are localto the tablet computer that can be used for game play and these clientdevices are included in the suggestion based on their profile.

Various ways to determine location of the primary client device may beengaged. For example, in one embodiment, for a wireless primary clientdevice, Wi-Fi signal strength to one or more multiple access pointswithin a home network or even access points of neighbors may be used todetermine the location within a user's house. In other embodiments, GPSdata may be used to determine position of the primary client device.Such position information, for example, could be used to determine if auser is entering a room in which another client device resides. The userentering the room could act as a trigger to test network connectivity ofthe other client device for streaming content. If the test identifiesthat the QOS of the other client device is really good, then a devicehandover option may be presented on the primary client device informingthe user that the other client device may be able to provide better gameplay experience—e.g., higher resolution, higher framerate, lowerlatency, etc., recommending the user to switch. User selection of theother client device at the option presented would make the switchhappen.

Oftentimes, the various personal client devices are continuouslyconnected to a network services, such as a gaming network, a socialmedia network, a content provider network, etc. Such network servicesmay keep track of the various personal client devices of a user, the geolocation of the client devices, and network connectivity. For instance,if a user is playing a game on a mobile phone using an online service,such as a gaming network, the online service may be able to determinelocation of the client device using signals generated by the mobilephone, such as GPS signal, cell tower signals, Wi-Fi hotspots, etc., andcommunicate with other client devices in a private or local network,such as a home network of the user. The other client devices in thelocal or private network (e.g., home network) may periodically connectto the gaming network or other online services and be made aware of thegeo location of the mobile phone of the user that is being used forplaying the game. If the gaming network detects that the user of themobile phone is close to home, this may initiate a trigger event tovalidate the network bandwidth and other QOS attributes of the mobilephone, as well as the network bandwidth of the other client devices inthe home network. This pre-establishment of device attributes of themobile phone and other client devices within a network may assist in afaster and frictionless device switch, if and when a user requests adevice handover, as the preliminary work to qualify the devices hasalready been done prior to receiving the switch request.

In some embodiments, a calendar event may be queried by an eventmechanism, for example, to identify an appointment that a user made foran online multiplayer session through a game network. The identificationof the calendar event may trigger preparation of the various clientdevices (primary and one or more secondary client devices) for astreaming session. Assuming that a client device is not being used forstreaming content yet, mere detection of a planned event could triggerprofiling and qualifying of the client devices (primary and secondaryclient devices) in a local network. This would include provisioningservices and attributes to get the primary and/or one or more secondaryclient devices ready for the streaming session ahead of time, and suchprovisioning is done in anticipation that at least one of the clientdevices will be selected for playing the game when the calendar eventoccurs.

The various embodiments describe ways to enable a current primary clientdevice to look for opportunities to reach out to all potential secondaryclient devices that a user may transfer the streaming session of gameplay. Particularly, the primary client device may engage in backgroundQOS and profiling of secondary client devices to obtain state of variousdevice attributes and such background QOS and profiling may be done atparticular times when the primary client device is detected to be idle,or on a fixed duty cycle, or periodically or in response to triggerevents, etc. The profiling is done before a user initiates a change,such that when and if the user decides to select a device handoveroption, a faster, frictionless change can occur as the secondary clientdevices have already been profiled and, in some embodiments, the codecsettings and/or other device attributes have been or can bepre-established for a quick stream switch.

If a particular secondary client device has a pattern of use, such usemay influence a pattern of QOS. By profiling the secondary clientdevices, the primary client device is made aware of such pattern of use,which can be helpful for qualifying the secondary client devices forswitching. For example, the profile may identify a secondary clientdevice having confidence intervals where the likely known QOS is high.This information can be used to invoke pre-tested codec and other streamsettings (i.e., device attributes that enable streaming of video framesor content), when and if the secondary client device is selected for adevice handover. In another example, if a secondary client device has anunpredictable process load (CPU load) during work hours, then new QOStesting may have to be performed every time a changeover event isinitiated.

In some embodiments the profiling information generated for the varioussecondary client devices may be used to determine if there are othersecondary client devices in the local or private or home network thatare already set up to receive streaming video frames of a game that arecurrently directed to a primary client device. The user may be providedwith a handover option with device identifiers of the secondary clientdevices that are already set up for receiving the streaming video framesof the game. In some embodiments, a new QOS test may be performed beforethe handover is performed. In alternate embodiments, the existing QOStest that was used to qualify the secondary client devices may be reliedon and the device handover effectuated without performing additional QOStest, so long as the secondary client devices and the primary clientdevice remain in the same network. In some embodiments, the primaryclient device may be any one of a personal computer, a tablet computer,a mobile phone, a desk top computer, a television, etc. The secondaryclient device may be a head mounted display. Alternately, the primaryclient device is a head mounted display and the secondary client deviceselected for device handover may be any one of a personal computer, atablet computer, a mobile phone, a desk top computer, a television, etc.In other embodiments, the primary client device and the secondary clientdevices may both be head mounted displays.

For more information on how progression of a user in the game is used tore-create the game state of the game, in one embodiment, reference canbe made to U.S. application Ser. No. 12/917,388, filed on Nov. 1, 2010,and titled, “USER INTERFACE, SYSTEM AND METHOD FOR CONTROLLING A VIDEOSTREAM,” which is incorporated herein by reference in its entirety. Theprocess of re-creating game state defined above is one possible way andthat other ways of re-creating the game state of the game may also beemployed. For more information on video generation and distribution,reference can be made to U.S. application Ser. No. 12/790,955, filed onMay 31, 2010, and titled, “GAME EXECUTION ENVIRONMENT,” (since issued asU.S. Pat. No. 8,506,402) which is incorporated herein by reference inits entirety. For more information related to server-side generationand/or rendering of game video, reference can be made to U.S.application Ser. No. 12/791,819, filed on Jun. 1, 2010, and titled“QUALIFIED VIDEO DELIVERY,” which is incorporated herein by reference inits entirety. For more information related to server-side execution ofcomputer programs, reference can be made to U.S. application Ser. No.13/231,751, filed on Sep. 13, 2011, and titled “ADD-ON MANAGEMENTSYSTEMS,” which is incorporated herein by reference in its entirety.

FIG. 7 illustrates the various method operations used for identifyingoptions of secondary client devices that can be used for a devicehandover of game play, in one embodiment of the invention. The methodbegins at operation 710, wherein a session of game play of a game isestablished for a primary client device, in response to a request forplaying the game received from the primary client device. The primaryclient device may be used to access a cloud gaming system via a useraccount to identify a game for game play. A server in the cloud gamingsystem authenticates the user and the game play request, and establishesthe game play session by executing an instance of the game for streamingvideo frames of the game to the primary client device. The video framesmay be encrypted using an encoder available on the server prior to beingstreamed to the primary client device.

A request is received to generate a profile of one or more secondaryclient devices, as illustrated in operation 720. The secondary clientdevices may be discovered by the primary client device using signalsgenerated by the secondary client devices and/or through one or moredevice discovery protocols implemented within the secondary clientdevices. The request for generating the profile may be initiated by theprimary client device by sending a signal to the secondary clientdevices to establish a communication connection with the server and getprofiled. Alternately, the primary client device may send a signal tothe server with device identifiers of the secondary client devices thatare discovered local to the primary client devices and request theserver to establish communication connection with the secondary clientdevices and exchange information to generate a profile of the respectivesecondary client devices. The server, in response to the signal,exchanges information with the secondary client devices using theestablished communication connection, identifies one or more deviceattributes and generates a profile of each of the secondary clientdevices using the respective device attributes. Information provided inthe profile is used to qualify the secondary client devices for gameplay of the game.

A handover option is provided to the primary client during the gameplay, as illustrated in operation 730. The handover option is configuredto identify the one or more secondary client devices that are qualifiedfor playing the game, based on the generated profile. The handoveroption is presented on a user interface at the primary client device foruser selection.

User selection of a secondary client device is received from the primaryclient device, as illustrated in operation 740. In response to receivingthe user selection of the secondary client device, the game play ispaused and a game state of the game play is saved by the server. Thepausing of game play leads to pausing streaming of video frames to theprimary client device. In response to pausing of the game, a resumeoption is provided on the secondary client device to resume playing ofthe game on the secondary client device.

User selection of a resume request from the secondary client device isreceived by the server, as illustrated in operation 750. In response tothe resume request, the server is configured to access the saved gamestate of the game and to continue the play of the game on the secondaryclient device. The resume request causes the server to stream videoframes of the resumed game to the secondary client device. The secondaryclient device is designated as the primary client device so that thestreaming video frames may be directed to the secondary client deviceinstead of the primary client device. The device handover is performedin a fast and seamless manner with minimal latency, enabling the user ofthe primary and secondary client device to enjoy comparable or enhancedgame play experience.

In some embodiments, the device switch may, depending on deviceattributes of the selected secondary client device, allow a user to havea better gameplay experience than what was provided by the primaryclient device, such as higher resolution, higher framerate, lowerlatency, etc. Alternately, the device switch may provide ways tocontinue game play without significant change in game play experience.This might be the case when the device switch is effectuated in responseto a trigger event, such as low battery power detected in the primaryclient device, for example.

Although the method operations were described in a specific order, itshould be understood that other housekeeping operations may be performedin between operations, or operations may be adjusted so that they occurat slightly different times, or may be distributed in a system whichallows the occurrence of the processing operations at various intervalsassociated with the processing, as long as the processing of the overlayoperations are performed in the desired way.

FIG. 8 illustrates an embodiment of an Information Service Providerarchitecture that may be used in providing access to different games.Information Service Providers (ISP) 1070 deliver a multitude ofinformation services to users 1082 geographically dispersed andconnected via network 1086. Although the various embodiments have beendiscussed with reference to providing fast access to games, theembodiments can be extended to provide one or more types of otherservices. For example, an ISP can deliver just one type of service, suchas a game, or a variety of services such as games, stock price updates,broadcast media, news, sports, gaming, etc. Additionally, the servicesoffered by each ISP may be dynamic, that is, services can be added ortaken away at any point in time. Thus, the ISP providing a particulartype of service to a particular individual can change over time. Forexample, a user may be served by an ISP in near proximity to the userwhile the user is in her home town, and the user may be served by adifferent ISP when the user travels to a different city. The home-townISP will transfer the required information and data from the user'sgaming or access profile to the new ISP through the connection module,such that the user information “follows” the user to the new city makingthe data closer to the user and easier to access. In another embodiment,a master-server relationship may be established between a master ISP,which manages the information for the user, and a server ISP thatinterfaces directly with the user under control from the master ISP. Inanother embodiment, the data is transferred from one ISP to another ISP(i.e., during switching of data center assigned to the user) as theclient moves around the world and such transfer may be based on acompatibility of services provided by the respective ISPs to make theISP in better position to service the user be the one that deliversthese services.

ISP 1070 includes Application Service Provider (ASP) 1072, whichprovides computer-based services to customers over a network. Softwareoffered using an ASP model is also sometimes called on-demand softwareor software as a service (SaaS). A simple form of providing access to aparticular application program (such as customer relationshipmanagement) is by using a standard protocol such as HTTP. Theapplication software resides on a vendor's system, for example, and isaccessed by users through a web browser using HTML, or by specialpurpose client software provided by the vendor, or via other remoteinterface such as a thin client.

Services delivered over a wide geographical area often use cloudcomputing. Cloud computing is a style of computing in which dynamicallyscalable and often virtualized resources are provided as a service overthe Internet. Users do not need to be an expert in the technologyinfrastructure in the “cloud” that supports them. Cloud computing can bedivided into different services, such as Infrastructure as a Service(IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).Cloud computing services often provide common business applicationsonline that are accessed from a web browser, while the software and dataare stored on the servers. The term cloud is used as a metaphor for theInternet (e.g., using servers, storage and logic), based on how theInternet is depicted in computer network diagrams and is an abstractionfor the complex infrastructure it conceals.

Further, ISP 1070 includes a Game Processing Server (GaPS) 1074 which isused by game clients to play single and multiplayer video games. Mostvideo games played over the Internet operate via a connection to a gameserver. Typically, games use a dedicated server application thatcollects data from players and distributes it to other players. This ismore efficient and effective than a peer-to-peer arrangement, but itrequires a separate server to host the server application. In anotherembodiment, the GaPS establishes communication between the players andtheir respective game-playing devices exchange information withoutrelying on the centralized GaPS.

Dedicated GaPSs are servers which run independently of the client. Suchservers are usually run on dedicated hardware located in data centers,providing more bandwidth and dedicated processing power. Dedicatedservers are the preferred method of hosting game servers for mostPC-based multiplayer games. Massively multiplayer online games run ondedicated servers usually hosted by the software company that owns thegame title, allowing them to control and update content.

Broadcast Processing Server (BPS) 1076 distributes audio or videosignals to an audience. Broadcasting to a very narrow range of audienceis sometimes called narrowcasting. The final leg of broadcastdistribution is how the signal gets to the listener or viewer, and itmay come over the air as with a radio station or TV station to anantenna and receiver, or may come through cable TV or cable radio (or“wireless cable”) via the station or directly from a network. TheInternet may also bring either radio or TV to the recipient, especiallywith multicasting allowing the signal and bandwidth to be shared.Historically, broadcasts have been delimited by a geographic region,such as national broadcasts or regional broadcast. However, with theproliferation of fast internet, broadcasts are not defined bygeographies as the content can reach almost any country in the world.

Storage Service Provider (SSP) 1078 provides computer storage space andrelated management services. SSPs also offer periodic backup andarchiving. By offering storage as a service, users can order morestorage as required. Another major advantage is that SSPs include backupservices and users will not lose all their data if their computers' harddrives fail. Further, a plurality of SSPs can have total or partialcopies of the user data, allowing users to access data in an efficientway independently of where the user is located or the device being usedto access the data. For example, a user can access personal files in thehome computer, as well as in a mobile phone while the user is on themove.

Communications Provider 1080 provides connectivity to the users. Onekind of Communications Provider is an Internet Service Provider (ISP)which offers access to the Internet. The ISP connects its customersusing a data transmission technology appropriate for delivering InternetProtocol datagrams, such as dial-up, DSL, cable modem, fiber, wirelessor dedicated high-speed interconnects. The Communications Provider canalso provide messaging services, such as e-mail, instant messaging, andSMS texting. Another type of Communications Provider is the NetworkService provider (NSP) which sells bandwidth or network access byproviding direct backbone access to the Internet. Network serviceproviders may consist of telecommunications companies, data carriers,wireless communications providers, Internet service providers, cabletelevision operators offering high-speed Internet access, etc.

Data Exchange 1088 interconnects the several modules inside ISP 1070 andconnects these modules to users 1082 via network 1086. Data Exchange1088 can cover a small area where all the modules of ISP 1070 are inclose proximity, or can cover a large geographic area when the differentmodules are geographically dispersed. For example, Data Exchange 1088can include a fast Gigabit Ethernet (or faster) within a cabinet of adata center, or an intercontinental virtual area network (VLAN).

Users 1082 access the remote services with client device 1084, whichincludes at least a CPU, a memory, a display and I/O. The client devicecan be a PC, a mobile phone, a netbook, tablet, gaming system, a PDA,etc. In one embodiment, ISP 1070 recognizes the type of device used bythe client and adjusts the communication method employed. In othercases, client devices use a standard communications method, such asHTML, to access ISP 1070.

FIG. 9 is a block diagram of a Game System 1400, according to variousembodiments of the invention. Game System 1400 is configured to providea video stream to one or more Clients 1410 via a Network 1415. TheNetwork is similar to the Network 200 illustrated in FIG. 1 . GameSystem 1400 typically includes a Video Server System 1420 and anoptional game server 1425. Video Server System 1420 is configured toprovide the video stream to the one or more Clients 1410 with a minimalquality of service. For example, Video Server System 1420 may receive agame command that changes the state of or a point of view within a videogame, and provide Clients 1410 with an updated video stream reflectingthis change instantly with minimal lag time. The Video Server System1420 may be configured to provide the video stream in a wide variety ofalternative video formats, including formats yet to be defined. Further,the video stream may include video frames configured for presentation toa user at a wide variety of frame rates. Typical frame rates are 30frames per second, 60 frames per second, and 1420 frames per second.Although higher or lower frame rates are included in alternativeembodiments of the invention.

Clients 1410, referred to herein individually as 1410A, 1410B, etc., mayinclude head mounted displays, terminals, personal computers, gameconsoles, tablet computers, telephones, set top boxes, kiosks, wirelessdevices, digital pads, stand-alone devices, handheld game playingdevices, and/or the like. The clients described are similar to clients100-1 through 100-n of FIG. 1 . Typically, Clients 1410 are configuredto receive encoded video streams, decode the video streams, and presentthe resulting video to a user, e.g., a player of a game. The processesof receiving encoded video streams and/or decoding the video streamstypically includes storing individual video frames in a receive bufferof the client. The video streams may be presented to the user on adisplay integral to Client 1410 or on a separate device such as amonitor or television. Clients 1410 are optionally configured to supportmore than one game player. For example, a game console may be configuredto support two, three, four or more simultaneous players. Each of theseplayers may receive a separate video stream, or a single video streammay include regions of a frame generated specifically for each player,e.g., generated based on each player's point of view. Clients 1410 areoptionally geographically dispersed. The number of clients included inGame System 1400 may vary widely from one or two to thousands, tens ofthousands, or more. As used herein, the term “game player” is used torefer to a person that plays a game and the term “game playing device”is used to refer to a device used to play a game. In some embodiments,the game playing device may refer to a plurality of computing devicesthat cooperate to deliver a game experience to the user. For example, agame console and an HMD may cooperate with the video server system 1420to deliver a game viewed through the HMD. In one embodiment, the gameconsole receives the video stream from the video server system 1420, andthe game console forwards the video stream, or updates to the videostream, to the HMD for rendering.

Clients 1410 are configured to receive video streams via Network 1415(similar to Network 200 of FIG. 1 ). Network 1415 may be any type ofcommunication network including, a telephone network, the Internet,wireless networks, powerline networks, local area networks, wide areanetworks, private networks, and/or the like. In typical embodiments, thevideo streams are communicated via standard protocols, such as TCP/IP orUDP/IP. Alternatively, the video streams are communicated viaproprietary standards.

A typical example of Clients 1410 is a personal computer comprising aprocessor, non-volatile memory, a display, decoding logic, networkcommunication capabilities, and input devices. The decoding logic mayinclude hardware, firmware, and/or software stored on a computerreadable medium. Systems for decoding (and encoding) video streams arewell known in the art and vary depending on the particular encodingscheme used.

Clients 1410 may, but are not required to, further include systemsconfigured for modifying received video. For example, a client may beconfigured to perform further rendering, to overlay one video image onanother video image, to crop a video image, and/or the like. Forexample, Clients 1410 may be configured to receive various types ofvideo frames, such as I-frames, P-frames and B-frames, and to processthese frames into images for display to a user. In some embodiments, amember of Clients 1410 is configured to perform further rendering,shading, conversion to 3-D, or like operations on the video stream. Amember of Clients 1410 is optionally configured to receive more than oneaudio or video stream. Input devices of Clients 1410 may include, forexample, a one-hand game controller, a two-hand game controller, agesture recognition system, a gaze recognition system, a voicerecognition system, a keyboard, a joystick, a pointing device, a forcefeedback device, a motion and/or location sensing device, a mouse, atouch screen, a neural interface, a camera, input devices yet to bedeveloped, and/or the like.

The video stream (and optionally audio stream) received by Clients 1410is generated and provided by Video Server System 1420. As is describedfurther elsewhere herein, this video stream includes video frames (andthe audio stream includes audio frames). The video frames are configured(e.g., they include pixel information in an appropriate data structure)to contribute meaningfully to the images displayed to the user. As usedherein, the term “video frames” is used to refer to frames includingpredominantly information that is configured to contribute to, e.g. toeffect, the images shown to the user. Most of the teachings herein withregard to “video frames” can also be applied to “audio frames.”

Clients 1410 are typically configured to receive inputs from a user.These inputs may include game commands configured to change the state ofthe video game or otherwise affect game play. The game commands can bereceived using input devices and/or may be automatically generated bycomputing instructions executing on Clients 1410. The received gamecommands are communicated from Clients 1410 via Network 1415 to VideoServer System 1420 and/or Game Server 1425. For example, in someembodiments, the game commands are communicated to Game Server 1425 viaVideo Server System 1420. In some embodiments, separate copies of thegame commands are communicated from Clients 1410 to Game Server 1425 andVideo Server System 1420. The communication of game commands isoptionally dependent on the identity of the command. Game commands areoptionally communicated from Client 1410A through a different route orcommunication channel that that used to provide audio or video streamsto Client 1410A.

Game Server 1425 is optionally operated by a different entity than VideoServer System 1420. For example, Game Server 1425 may be operated by thepublisher of a multiplayer game. In this example, Video Server System1420 is optionally viewed as a client by Game Server 1425 and optionallyconfigured to appear from the point of view of Game Server 1425 to be aprior art client executing a prior art game engine. Communicationbetween Video Server System 1420 and Game Server 1425 optionally occursvia Network 1415. As such, Game Server 1425 can be a prior artmultiplayer game server that sends game state information to multipleclients, one of which is Video Server System 1420. Video Server System1420 may be configured to communicate with multiple instances of GameServer 1425 at the same time. For example, Video Server System 1420 canbe configured to provide a plurality of different video games todifferent users. Each of these different video games may be supported bya different Game Server 1425 and/or published by different entities. Insome embodiments, several geographically distributed instances of VideoServer System 1420 are configured to provide game video to a pluralityof different users. Each of these instances of Video Server System 1420may be in communication with the same instance of Game Server 1425.Communication between Video Server System 1420 and one or more GameServer 1425 optionally occurs via a dedicated communication channel. Forexample, Video Server System 1420 may be connected to Game Server 1425via a high bandwidth channel that is dedicated to communication betweenthese two systems.

Video Server System 1420 comprises at least a Video Source 1430, an I/ODevice 1445, a Processor 1450, and Storage 1455 (includingnon-transitory analog and/or digital storage devices). Video ServerSystem 1420 may include one computing device or be distributed among aplurality of computing devices. These computing devices are optionallyconnected via a communications system such as a local area network.

Video Source 1430 is configured to provide a video stream, e.g.,streaming video or a series of video frames that form a moving picture.In some embodiments, Video Source 1430 includes a video game engine andrendering logic. The video game engine is configured to receive gamecommands from a player and to maintain a copy of the state of the videogame based on the received commands. This game state includes theposition of objects in a game environment, as well as typically a pointof view. The game state may also include properties, images, colorsand/or textures of objects.

The game state is typically maintained based on game rules, as well asgame commands such as move, turn, attack, set focus to, interact, use,and/or the like. Part of the game engine is optionally disposed withinGame Server 1425. Game Server 1425 may maintain a copy of the state ofthe game based on game commands received from multiple players usinggeographically disperse clients. In these cases, the game state isprovided by Game Server 1425 to Video Source 1430, wherein a copy of thegame state is stored and rendering is performed. Game Server 1425 mayreceive game commands directly from Clients 1410 via Network 1415,and/or may receive game commands via Video Server System 1420.

Video Source 1430 typically includes rendering logic, e.g., hardware,firmware, and/or software stored on a computer readable medium such asStorage 1455. This rendering logic is configured to create video framesof the video stream based on the game state. All or part of therendering logic is optionally disposed within a graphics processing unit(GPU). Rendering logic typically includes processing stages configuredfor determining the three-dimensional spatial relationships betweenobjects and/or for applying appropriate textures, etc., based on thegame state and viewpoint. The rendering logic produces raw video that isthen usually encoded prior to communication to Clients 1410. Forexample, the raw video may be encoded according to an Adobe Flash®standard, .wav, H.264, H.263, On2, VP6, VC-1, WMA, Huffyuv, Lagarith,MPG-x. Xvid. FFmpeg, x264, VP6-8, realvideo, mp3, or the like. Theencoding process produces a video stream that is optionally packaged fordelivery to a decoder on a remote device. The video stream ischaracterized by a frame size and a frame rate. Typical frame sizesinclude 800×600, 1280×720 (e.g., 720p), 1024×768, although any otherframe sizes may be used. The frame rate is the number of video framesper second. A video stream may include different types of video frames.For example, the H.264 standard includes a “P” frame and an “I” frame.I-frames include information to refresh all macro blocks/pixels on adisplay device, while P-frames include information to refresh a subsetthereof. P-frames are typically smaller in data size than are I-frames.As used herein the term “frame size” is meant to refer to a number ofpixels within a frame. The term “frame data size” is used to refer to anumber of bytes required to store the frame.

In alternative embodiments, Video Source 1430 includes a video recordingdevice such as a camera. This camera may be used to generate delayed orlive video that can be included in the video stream of a computer game.The resulting video stream optionally includes both rendered images andimages recorded using a still or video camera. Video Source 1430 mayalso include storage devices configured to store previously recordedvideo to be included in a video stream. Video Source 1430 may alsoinclude motion or positioning sensing devices configured to detectmotion or position of an object, e.g., person, and logic configured todetermine a game state or produce video-based on the detected motionand/or position.

Video Source 1430 is optionally configured to provide overlaysconfigured to be placed on other video. For example, these overlays mayinclude a command interface, log in instructions, messages to a gameplayer, images of other game players, video feeds of other game players(e.g., webcam video). In embodiments of Client 1410A including a touchscreen interface or a gaze detection interface, the overlay may includea virtual keyboard, joystick, touch pad, and/or the like. In one exampleof an overlay a player's voice is overlaid on an audio stream. VideoSource 1430 optionally further includes one or more audio sources.

In embodiments wherein Video Server System 1420 is configured tomaintain the game state based on input from more than one player, eachplayer may have a different point of view comprising a position anddirection of view. Video Source 1430 is optionally configured to providea separate video stream for each player based on their point of view.Further, Video Source 1430 may be configured to provide a differentframe size, frame data size, and/or encoding to each of Client 1410.Video Source 1430 is optionally configured to provide 3-D video.

I/O Device 1445 is configured for Video Server System 1420 to sendand/or receive information such as video, commands, requests forinformation, a game state, gaze information, device motion, devicelocation, user motion, client identities, player identities, gamecommands, security information, audio, and/or the like. I/O Device 1445typically includes communication hardware such as a network card ormodem. I/O Device 1445 is configured to communicate with Game Server1425, Network 1415, and/or Clients 1410.

Processor 1450 is configured to execute logic, e.g. software, includedwithin the various components of Video Server System 1420 discussedherein. For example, Processor 1450 may be programmed with softwareinstructions in order to perform the functions of Video Source 1430,Game Server 1425, and/or a Client Qualifier 1460. Video Server System1420 optionally includes more than one instance of Processor 1450.Processor 1450 may also be programmed with software instructions inorder to execute commands received by Video Server System 1420, or tocoordinate the operation of the various elements of Game System 1400discussed herein. Processor 1450 may include one or more hardwaredevice. Processor 1450 is an electronic processor.

Storage 1455 includes non-transitory analog and/or digital storagedevices. For example, Storage 1455 may include an analog storage deviceconfigured to store video frames. Storage 1455 may include a computerreadable digital storage, e.g. a hard drive, an optical drive, or solidstate storage. Storage 1455 is configured (e.g. by way of an appropriatedata structure or file system) to store video frames, artificial frames,a video stream including both video frames and artificial frames, audioframe, an audio stream, and/or the like. Storage 1455 is optionallydistributed among a plurality of devices. In some embodiments, Storage1455 is configured to store the software components of Video Source 1430discussed elsewhere herein. These components may be stored in a formatready to be provisioned when needed.

Video Server System 1420 optionally further comprises Client Qualifier1460. Client Qualifier 1460 is configured for remotely determining thecapabilities of a client, such as Clients 1410A or 1410B. Thesecapabilities can include both the capabilities of Client 1410A itself aswell as the capabilities of one or more communication channels betweenClient 1410A and Video Server System 1420. For example, Client Qualifier1460 may be configured to test a communication channel through Network1415.

Client Qualifier 1460 can determine (e.g., discover) the capabilities ofClient 1410A manually or automatically. Manual determination includescommunicating with a user of Client 1410A and asking the user to providecapabilities. For example, in some embodiments, Client Qualifier 1460 isconfigured to display images, text, and/or the like within a browser ofClient 1410A. In one embodiment, Client 1410A is an HMD that includes abrowser. In another embodiment, client 1410A is a game console having abrowser, which may be displayed on the HMD. The displayed objectsrequest that the user enter information such as operating system,processor, video decoder type, type of network connection, displayresolution, etc. of Client 1410A. The information entered by the user iscommunicated back to Client Qualifier 1460.

Automatic determination may occur, for example, by execution of an agenton Client 1410A and/or by sending test video to Client 1410A. The agentmay comprise computing instructions, such as java script, embedded in aweb page or installed as an add-on. The agent is optionally provided byClient Qualifier 1460. In various embodiments, the agent can find outprocessing power of Client 1410A, decoding and display capabilities ofClient 1410A, lag time reliability and bandwidth of communicationchannels between Client 1410A and Video Server System 1420, a displaytype of Client 1410A, firewalls present on Client 1410A, hardware ofClient 1410A, software executing on Client 1410A, registry entrieswithin Client 1410A, and/or the like.

Client Qualifier 1460 includes hardware, firmware, and/or softwarestored on a computer readable medium. Client Qualifier 1460 isoptionally disposed on a computing device separate from one or moreother elements of Video Server System 1420. For example, in someembodiments, Client Qualifier 1460 is configured to determine thecharacteristics of communication channels between Clients 1410 and morethan one instance of Video Server System 1420. In these embodiments theinformation discovered by Client Qualifier can be used to determinewhich instance of Video Server System 1420 is best suited for deliveryof streaming video to one of Clients 1410.

With the above embodiments in mind, it should be understood that theinvention may employ various computer-implemented operations involvingdata stored in computer systems. These operations include operationsrequiring physical manipulation of physical quantities. Any of theoperations described herein that form part of the invention are usefulmachine operations. The invention also relates to a device or anapparatus for performing these operations. The apparatus can bespecially constructed for the required purpose, or the apparatus can bea general-purpose computer selectively activated or configured by acomputer program stored in the computer. In particular, variousgeneral-purpose machines can be used with computer programs written inaccordance with the teachings herein, or it may be more convenient toconstruct a more specialized apparatus to perform the requiredoperations.

The above described invention may be practiced with other computersystem configurations including hand-held devices, microprocessorsystems, microprocessor-based or programmable consumer electronics,minicomputers, mainframe computers and the like. The invention may alsobe practiced in distributing computing environments where tasks areperformed by remote processing devices that are linked through acommunications network.

The invention can also be embodied as computer readable programming codeon a computer-readable medium. Alternately, the computer readableprogramming code may be downloaded from a server using the data exchangeinterconnects described above. The computer-readable medium is any datastorage device that can store data which can be thereafter read by acomputer system, including an electromagnetic wave carrier. Examples ofthe computer-readable medium include hard drives, network attachedstorage (NAS), read-only memory, random-access memory, CD-ROMs, CD-Rs,CD-RWs, magnetic tapes, and other optical and non-optical data storagedevices. The computer-readable medium can also be distributed over anetwork coupled computer system so that the computer readable code isstored and executed in a distributed fashion.

Although the foregoing invention has been described in some detail forpurposes of clarity of understanding, it will be apparent that certainchanges and modifications may be practiced within the scope of theappended claims. Accordingly, the present embodiments are to beconsidered as illustrative and not restrictive, and the invention is notto be limited to the details given herein, but may be modified withinthe scope and equivalents of the appended claims.

What is claimed is:
 1. A method, comprising: establishing a session ofgame play of a game, the game play generating game content that isreturned to a primary client device of a user for rendering; generatingdevice profile of each secondary client device of a plurality ofsecondary client devices associated with the user, wherein said eachsecondary client device is identified to be local to the primary clientdevice, the device profile identifying device attributes of said eachsecondary client device; identifying one or more of the secondary clientdevices from the plurality of secondary client devices with devicecapabilities for presenting a component of the game content, the one ormore secondary client devices identified by matching the deviceattributes of said each of the one or more secondary client device todevice attribute requirements for rendering said component of the game;providing a handover option to the primary client device, the handoveroption including a list of the one or more of the secondary clientdevices that is capable of rendering said component of the game content;and receiving selection of a secondary client device from the list, theselection causing automatic establishment of communication connectionbetween the primary client device and the selected secondary clientdevice, the establishing of communication connection includesautomatically switching streaming of said component of the game contentto the selected secondary client device for rendering and continuingrendering of remaining ones of components of the game content at theprimary client device.
 2. The method of claim 1, wherein generation ofthe device profile is initiated at the primary client device, thegenerating of device profile further including, detecting one or moresecondary client devices that are local to the primary client device,the detecting is in response to a seeking signal initiated by theprimary client device actively seeking the secondary client devices thatare local to the primary client device; and forwarding a profile signalto each secondary client device that is detected to be local to theprimary client device, the profile signal including a request to saidsecondary client device to get profiled by a server of a cloud gamingsite, the profile of said secondary client device identifying devicecapabilities for using to qualify the secondary client devices to playthe game or to render game content of the game.
 3. The method of claim2, wherein the server used by the secondary client device to getprofiled is a first server that is executing an instance of the game. 4.The method of claim 2, wherein the server used by the secondary clientdevice to get profile is a second server that is different from a firstserver that is executing an instance of the game, wherein the secondserver is within a first data center where the first server is locatedor is in a second data center, the second server configured to performprofiling of the secondary client device.
 5. The method of claim 1,wherein generation of the device profile is initiated at each secondaryclient device, wherein the generating of device profile furtherincluding, receiving a profile request from said each secondary clientdevice that is local to the primary client device to get profiled by aserver of a cloud gaming site, the request initiated by each secondaryclient device upon detecting the secondary client device local to theprimary client device.
 6. The method of claim 1, wherein said componentthat is automatically switched to the selected secondary client deviceis an audio component, or a video component, or select audio data fromthe audio component or select video data from the video component. 7.The method of claim 1, wherein identifying the one or more secondaryclient devices is in response to a trigger event detected at the primaryclient device, and wherein the trigger event is initiated upon detectingchange in one or more device attributes of the primary client devicethat prevents optimal rendering of the component at the primary clientdevice.
 8. The method of claim 1, further including, evaluating thedevice attributes of the secondary client device during streaming of thecomponent the secondary client device, the evaluating is to determine ifthe device attributes of the secondary client device has changed duringrendering of the component; and when the device attributes of thesecondary client device are insufficient for rendering the component,automatically provisioning resources so as to cause the deviceattributes of the secondary client device to meet the device attributerequirements for rendering the component of the game.
 9. The method ofclaim 1, wherein automatically switching streaming of said component ofthe game content to the selected secondary client device furtherincluding, pausing rendering of said component of the game at theprimary client device; and resuming rendering of said component at thesecondary client device.
 10. The method of claim 1, whereinautomatically switching streaming of said component of the game contentto the selected secondary client device further including setting saidsecondary client device as default client device for rendering saidcomponent.
 11. A cloud gaming system, comprising: a server configured toexecute a plurality of games, the server including, a game executionengine configured to identify game code of a game selected for playingat a first client device associated with a user account, and execute thegame code to generate streams of video frames for the game; a clientdevice profiling module configured to, generate device profile of eachsecond client device of a plurality of second client devices that arelocal to the first client device, the device profile identifying deviceattributes of said second client device; identify one or more secondclient devices from the plurality of second client devices with devicecapabilities for presenting a component of the game content, the one ormore second client devices identified by matching the device attributesof said second client device to device attribute requirements forrendering the component of the game content; provide a handover optionto the first client device, the handover option including a list of theone or more of the second client devices that is capable for renderingsaid component of the game content; and a handover manager configuredto, receive selection of a second client device from the list includedin the handover option; and responsive to selection, establishcommunication connection between the selected second client device andthe first client device, the establishing of communication connectionincludes automatically switching streaming of said component of the gamecontent to the selected second client device for rendering andcontinuing rendering of remaining ones of components of the game contentat the first client device.
 12. The cloud gaming system of claim 11,wherein the handover manager is configured to, communicate a pausesignal to a pause manager to stop streaming the component of the gamecontent to the first client device; and communicate a resume signal to aresume manager to start streaming the component of the game content tothe second client device.
 13. The cloud gaming system of claim 12,wherein the pause signal to the pause manager and the resume signal tothe resume manager is in response to the trigger event detected.
 14. Thecloud gaming system of claim 12, wherein the game execution engine isconfigured to communicate with a state data manager to obtain a pointfrom where the component of game content is to be resumed at the secondclient device, based on game state of the game.
 15. The cloud gamingsystem of claim 11, wherein the handover manager is configured to updatedevice identifier at the game execution engine from a first clientdevice identifier to a second client device identifier, in response tothe switch in the streaming of the component of the game content.